Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Imgee V2 #59

Merged
merged 43 commits into from
Jul 17, 2017
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
441629f
removed celery as dependency
Apr 7, 2017
7337d16
fixed - was missing folder name when downloading file
Apr 10, 2017
ba53794
handling multiple resize request for same file
Apr 11, 2017
412e44b
removed print statements
Apr 11, 2017
9a49863
if there is an error while processing a file, raise it
Apr 11, 2017
127da69
fixed comment
Apr 11, 2017
be447ac
removed all tests
Apr 11, 2017
3b54fbd
removed more unnecessary things
Apr 11, 2017
6cd46ce
removed unnecessary imports and minor cleanup
Apr 11, 2017
57a99f0
renamed async.py
Apr 11, 2017
643e476
Merge pull request #56 from hasgeek/celery-removal
shreyas-satish Apr 12, 2017
9ab5a45
added label tests
Apr 14, 2017
d57fec0
separating label logic
Apr 14, 2017
e0f547f
fixed label tests
Apr 14, 2017
cb7b9bd
more cleanup, moved some views out of index.py
Apr 17, 2017
c6c4645
cleaned up label saving codes a little
Apr 17, 2017
b3ff9f4
added tests for upload and delete
Apr 17, 2017
c663086
updated status code for delete test
Apr 17, 2017
e4ae277
mixed couple of migrations and showing error
Apr 17, 2017
eb5cab0
Merge pull request #57 from hasgeek/new-tests
shreyas-satish Apr 17, 2017
b4a14c1
some more cleanups
Apr 18, 2017
6fabf71
Merge pull request #58 from hasgeek/new-tests
shreyas-satish Apr 18, 2017
fc28fea
using get_image_url to create image url
Apr 18, 2017
edc1bf9
not showing delete label button if not logged in
Apr 18, 2017
95c1615
importing only the needed forms
Apr 18, 2017
95adecf
added tests for resizing files and testing with multiple file types
Apr 20, 2017
4451fb0
comparing redirect location with generated thumbnail url
Apr 20, 2017
4635676
added redis to travis
Apr 20, 2017
838d1ec
fixed conflicts
May 4, 2017
418e080
Merge branch 'master' of github.com:hasgeek/imgee into imgee-v2
May 9, 2017
f112e6c
fixed fixtures file to use new init_app behavior
May 9, 2017
cc369e6
fixed conflict
Jun 1, 2017
380bb87
Function name fix, using uuid1mc, time format fix, TaskRegistry fix (…
Jun 23, 2017
f2c6fd6
fixed conflict
Jun 23, 2017
23007ec
lot of minor fixes
Jun 29, 2017
59bb581
using dotted relative import syntax everywhere, added tests for regis…
Jun 30, 2017
d31b027
cleaning up queries and imports
Jul 5, 2017
ff390de
cleanup of os.path and more comments on thumbnail
Jul 13, 2017
6731ef9
fixed typo
Jul 13, 2017
3dcb9e5
added buildspec.yml file for testing AWS CodeBuild
Jul 13, 2017
d2c1228
setting env varialbe manually for aws codebuild
Jul 13, 2017
88457f5
removed buildspec.yml file
Jul 17, 2017
102e603
showing loading spinner if file is still processing
Jul 17, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,7 @@ baseframe-packed.css
baseframe-packed.js
error.log
imgee/static/uploads
*.bak
instance/production.env.sh
imgee/static/gen
tests/imgee/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What goes in this folder?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uploads made during running the tests.

-> % tree tests/imgee 
tests/imgee
└── static
    └── test_uploads
        ├── 0a795f311f90494292df03f64317c57d.svg
        ├── 1ce47cefeec241b9a7266ed852131e48.svg
        ├── 28029c34ba9947ad8e6ce53e074b3617.gif
        ├── 318c1dbbefa7451e82fee1d7c77e3282.svg
        ├── 31b9a8a9be6947db9913fdb38c0fd4e2.jpeg

12 changes: 7 additions & 5 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
env:
global:
env:
global:
- secure: |-
G5Dn+zkbN/BeNoopxtM2idC2Hy1ebJxRxprD7XEAmH6VO26ANgWYLI1dMrdH
uQc9F317STawQ/Um6KnqjErOKkC2BUYTOTj8AzoPVFz6NcK/Ca4d9Vfbtf5u
Expand All @@ -9,13 +9,15 @@ env:
EGeHYvQUiyAg7zM4KdNJr6txj+jBE8MAeh7EwYNHoh9B7Vx//GxmXFnWyjXV
9cJkFroDW1Zfs2SZjLtzMQC8YXE30jmMxg+XHCQewKzRa4u1320=
language: python
python:
python:
- 2.7
services:
- redis-server
before_script:
- mkdir imgee/static/test_uploads
script:
script:
- nosetests --with-coverage
install:
install:
- sudo apt-get -qq install zlib1g-dev libfreetype6-dev liblcms1-dev libwebp-dev libjpeg-dev libpng-dev libfreetype6-dev libtiff4-dev librsvg2-dev ghostscript imagemagick pandoc
- pip install -r requirements.txt
- pip install nose coverage BeautifulSoup4 Pillow
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are BS and Pillow only required for tests?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, no. Neither of them are being used now. Removing.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,14 @@
def upgrade():
op.add_column('thumbnail', sa.Column('width', sa.Integer(), nullable=True))
op.add_column('thumbnail', sa.Column('height', sa.Integer(), nullable=True))

connection = op.get_bind()
tn = Thumbnail.__table__
result = connection.execute(select([column('id'), column('size')], from_obj=tn))
w_h = [dict(tnid=r.id, w=int(r.size.split('x')[0]), h=int(r.size.split('x')[1])) for r in result]
updt_stmt = tn.update().where(tn.c.id == bindparam('tnid')).values(width=bindparam('w'), height=bindparam('h'))
connection.execute(updt_stmt, w_h)
if len(w_h) > 0:
updt_stmt = tn.update().where(tn.c.id == bindparam('tnid')).values(width=bindparam('w'), height=bindparam('h'))
connection.execute(updt_stmt, w_h)


def downgrade():
Expand Down
5 changes: 3 additions & 2 deletions alembic/versions/2c7e25599132_add_mimetype.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,9 @@ def upgrade():
sf = StoredFile.__table__
result = connection.execute(select([sf.c.id, sf.c.title]))
mimetypes = [dict(sfid=r[0], mimetype=guess_type(r[1])[0]) for r in result]
updt_stmt = sf.update().where(sf.c.id == bindparam('sfid')).values(mimetype=bindparam('mimetype'))
connection.execute(updt_stmt, mimetypes)
if len(mimetypes) > 0:
updt_stmt = sf.update().where(sf.c.id == bindparam('sfid')).values(mimetype=bindparam('mimetype'))
connection.execute(updt_stmt, mimetypes)


def downgrade():
Expand Down
3 changes: 1 addition & 2 deletions alembic/versions/2d5db2a698f6_init.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,6 @@ def upgrade():
sa.Column('created_at', sa.DateTime(), nullable=False),
sa.Column('updated_at', sa.DateTime(), nullable=False),
sa.Column('userid', sa.Unicode(length=22), nullable=False),
sa.Column('description', sa.UnicodeText(), nullable=False),
sa.Column('type', sa.Integer(), nullable=False),
sa.Column('name', sa.Unicode(length=250), nullable=False),
sa.Column('title', sa.Unicode(length=250), nullable=False),
sa.PrimaryKeyConstraint('id'),
Expand All @@ -63,6 +61,7 @@ def upgrade():
sa.Column('created_at', sa.DateTime(), nullable=False),
sa.Column('updated_at', sa.DateTime(), nullable=False),
sa.Column('name', sa.String(length=50), nullable=False),
sa.Column('title', sa.String(length=50), nullable=False),
sa.Column('profile_id', sa.Integer(), nullable=False),
sa.ForeignKeyConstraint(['profile_id'], ['profile.id'], ),
sa.PrimaryKeyConstraint('id')
Expand Down
11 changes: 9 additions & 2 deletions alembic/versions/347ba3ac054f_update_image_attrs.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@
import os.path
import sys
from glob import glob
from alembic import op
from sqlalchemy.sql import select
from sqlalchemy.orm.session import sessionmaker
from sqlalchemy.orm import load_only

sys.path.append('../../')
from imgee import db
Expand All @@ -22,7 +26,10 @@


def upgrade():
imgs = StoredFile.query.filter_by(size=None)
connection = op.get_bind()
Session = sessionmaker(bind=connection.engine)
session = Session(bind=connection)
imgs = session.query(StoredFile).filter_by(size=None).options(load_only("id", "name", "title"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i believe Alembic gives you a session already. Is a new session necessary?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jace can do that with op.execute() but then have to define the schema for stored file in this migration file itself. I wanted to avoid it. so creating a session and using that so that I can use the ORM.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a problem. Migrations must never import models. They must always be locally defined in the migration.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason is that the model may no longer exist in the same form in the code when the migration is run. You need version controlling.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jace that's exactly why I did this. The way it was before, the model was imported and used which was breaking the migrations. The way above, it uses the session of the migration, so it uses the version of the model at that point in migration and not the current version of the model.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That doesn't make sense. You're importing from models. You always get the current version of the model.


for img in imgs:
path = path_for(img.name) + '.*'
Expand All @@ -34,7 +41,7 @@ def upgrade():
print 'updated attributes of %s\n' % img.title,
else:
print 'local file not found for %s\n' % img.title,
db.session.commit()
session.commit()


def downgrade():
Expand Down
13 changes: 4 additions & 9 deletions imgee/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
# The imports in this file are order-sensitive

import os
from celery import Celery

from flask import Flask, redirect, url_for
from flask.ext.lastuser import Lastuser
Expand All @@ -12,20 +11,18 @@
import coaster.app
from ._version import __version__


version = Version(__version__)
app = Flask(__name__, instance_relative_config=True)
lastuser = Lastuser()
celery = Celery()

assets['imgee.css'][version] = 'css/app.css'

from . import models, views
from .models import db
from .api import api
from .async import TaskRegistry
from .tasks import TaskRegistry

registry = TaskRegistry(os.getenv('ENV', 'production'))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any dependency on system environment should be in coaster.app. The system environment is not a secure place for keeping settings.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can I set this inside the init_for(env) function and then use env? TaskRegistry has name='default' set by default. I wanted to use something more verbose for the redis keys. hence did this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hardcode name='imgee'?


registry = TaskRegistry()

def mkdir_p(dirname):
if not os.path.exists(dirname):
Expand All @@ -40,14 +37,12 @@ def error403(error):
def init_for(env):
coaster.app.init_app(app, env)
baseframe.init_app(app, requires=['baseframe', 'picturefill', 'imgee'])
app.error_handlers[403] = error403
app.error_handlers[403] = error403
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this Flask's recommended way of defining an error handler? If not, we should use the recommended way.

lastuser.init_app(app)
lastuser.init_usermanager(UserManager(db, models.User))
if app.config.get('MEDIA_DOMAIN') and (
app.config['MEDIA_DOMAIN'].startswith('http:') or
app.config['MEDIA_DOMAIN'].startswith('https:')):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the docs, startswith takes a tuple. You can use: app.config['MEDIA_DOMAIN'].startswith(('http:', 'https:'))

app.config['MEDIA_DOMAIN'] = app.config['MEDIA_DOMAIN'].split(':', 1)[1]
mkdir_p(app.config['UPLOADED_FILES_DEST'])
celery.conf.add_defaults(app.config)
registry.set_connection()
app.register_blueprint(api, url_prefix='/api/1')
60 changes: 0 additions & 60 deletions imgee/api.py

This file was deleted.

107 changes: 0 additions & 107 deletions imgee/async.py

This file was deleted.

12 changes: 6 additions & 6 deletions imgee/forms.py
Original file line number Diff line number Diff line change
@@ -1,19 +1,17 @@
# -*- coding: utf-8 -*-

import os.path
from coaster import make_name
from flask.ext.wtf import Form
from baseframe.forms import Form
from wtforms.validators import Required, ValidationError, Length
from wtforms import (FileField, TextField, HiddenField,
SelectMultipleField, SelectField)
from wtforms import (FileField, TextField, HiddenField, SelectField)

from imgee import app
from imgee.models import Label
from imgee.utils import get_file_type, is_file_allowed
from imgee.utils import is_file_allowed


def valid_file(form, field):
if not is_file_allowed(field.data.stream):
if not is_file_allowed(field.data.stream, field.data.mimetype, field.data.filename):
raise ValidationError("Sorry, unknown image format. Please try uploading another file.")


Expand All @@ -28,6 +26,7 @@ class DeleteImageForm(Form):
class PurgeCacheForm(Form):
pass


def reserved_words():
"""get all words which can't be used as labels"""
words = []
Expand Down Expand Up @@ -67,6 +66,7 @@ class EditTitleForm(Form):
file_name = HiddenField('file_name')
file_title = TextField('title', validators=[Required(), Length(max=250)])


class UpdateTitle(Form):
title = TextField('Title', validators=[Required(), Length(max=250)])

Expand Down
Loading