Migrate boto -> boto3

[hershaw]

Info

boto maintenance has staled and although boto is python 3 compatible system-wide boto plugins may not be which can cause an import of SFrame objects to fail at runtime. For example, this will happen on GCE ubuntu machines

boto is used in two places: file_util.py and test_gl_pickler.py

file_util.py is unit tested here

TODO

• Change setup.py to remove boto and include boto3
• Migrate file_util.py
  • TODO: spec boto3 api usage
• Migrate test_gl_picker.py
  • TODO: spec boto3 api usage

[hershaw]

In my fork I have changed setup.py to use boto3 and then re-built the package:

source oss_local_scripts/python_env.sh debug
cd debug/oss_src/unity/python
make -j 4

But when I try to do an import of boto3 in file_util.py it's not available. Do I need to rebuild the miniconda package or something using the new setup.py?

@TobyRoseman

[TobyRoseman]

@hershaw - Good question. Changes to the setup.py affect the egg/wheel but not dev builds, which is what you're running. To change that, make the same change here: https://github.com/dato-code/SFrame/blob/fac7e34200e9105e9f54ab46d60c12a7643d070b/oss_local_scripts/pip_requirements.txt

Then you'll need to do a configure --cleanup. Then rerun configure and rerun make.

Let me know if you have any more questions.