Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add read mode for dxpy #138

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

add read mode for dxpy #138

wants to merge 5 commits into from

Conversation

ying-w
Copy link
Contributor

@ying-w ying-w commented Aug 15, 2020

Now that dxpy supports non-utf8 read modes, pass 'rb' through

make lint doesn't work tho:

Warning, treated as error:
failed to reach any of the inventories with the following issues:
intersphinx inventory 'http://docs.python.org/3/objects.inv' not fetchable due to <class 'requests.exceptions.ConnectionError'>: HTTPConnectionPool(host='docs.python.org', port=80): Max retries exceeded with url: /3/objects.inv (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x107478d50>: Failed to establish a new connection: [Errno 60] Operation timed out'))

@@ -128,7 +128,7 @@ def normpath(self):
normed = posixpath.normpath('/' + str(self)[len(self.drive):])[1:]
return self.path_class(self.drive + normed)

def read_object(self):
def read_object(self, mode='r'):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the intention is to use with stor.Path('dx://path/to/file).open(mode='rb') to access dx files, this is not really necessary because mode is already set in the self object by this point. You can remove it from dx.py, s3.py and swift.py as well. You'd just do in dx.py:

def read_object(self):
    ....
    file_handler = dxpy.DXFile(dxid=self.canonical_resource, project=self.canonical_project, mode=self.mode)
    ....
    if self.mode == 'r':
        ....

If there was some other reasoning behind this, you can disregard my comment.

Copy link
Contributor Author

@ying-w ying-w Aug 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback. I just realized that OBSFile defined mode i was looking for it in OBSPath instead and not finding it so I dont think the self.mode in your example would work

I'll make the change but I'm not really sure about when when to use open() vs read_object()

@jtratner
Copy link
Contributor

I think that the correct answer is to just not decode? Probably just need to review this again - good point...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants