Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: improve experience with dockerized tests and runtime execution #12

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

marcellodesales
Copy link

@marcellodesales marcellodesales commented Mar 8, 2023

🎉 Improvements

DevOps: Docker becomes the development interface: dev builds the docker image, while ops deploys and runs it as a container.

🐛 Bug fixes

  • 6b40e58: Fix the execution of test cases in clean environment when the dir /var/minitwit does not exist
    • See commit message for details
self = <sqlalchemy.dialects.sqlite.pysqlite.SQLiteDialect_pysqlite object at 0x4009bb1040>, cargs = ('/var/minitwit/minitwit.db',), cparams = {}

    def connect(self, *cargs, **cparams):
        # inherits the docstring from interfaces.Dialect.connect
>       return self.dbapi.connect(*cargs, **cparams)
E       sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
E       (Background on this error at: https://sqlalche.me/e/14/e3q8)

/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py:598: OperationalError

🐳 Dockerized Execution

✅ Tests are executed from the docker container

$ docker compose up --build minitwit-test
[+] Building 21.8s (18/18) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                     0.1s
 => => transferring dockerfile: 1.56kB                                                                                                                   0.0s
 => [internal] load .dockerignore                                                                                                                        0.0s
 => => transferring context: 2B                                                                                                                          0.0s
 => [internal] load metadata for docker.io/library/python:3.8.16-alpine3.17                                                                              1.2s
 => [auth] library/python:pull token for registry-1.docker.io                                                                                            0.0s
 => [internal] load build context                                                                                                                        0.0s
 => => transferring context: 490B                                                                                                                        0.0s
 => [builder 1/9] FROM docker.io/library/python:3.8.16-alpine3.17@sha256:8518dd6657131d938f283ea97385b1db6724e35d45ddab6cd1c583796e35566a                0.0s
 => => resolve docker.io/library/python:3.8.16-alpine3.17@sha256:8518dd6657131d938f283ea97385b1db6724e35d45ddab6cd1c583796e35566a                        0.0s
 => CACHED [builder 2/9] RUN apk add musl-dev python3-dev mariadb-dev gcc build-base bash                                                                0.0s
 => CACHED [builder 3/9] WORKDIR /viasat/minitwit                                                                                                        0.0s
 => CACHED [builder 4/9] COPY requirements.txt .                                                                                                         0.0s
 => CACHED [builder 5/9] RUN pip install -r requirements.txt                                                                                             0.0s
 => CACHED [builder 6/9] COPY ./static /viasat/minitwit/static                                                                                           0.0s
 => CACHED [builder 7/9] COPY ./templates /viasat/minitwit/templates                                                                                     0.0s
 => CACHED [builder 8/9] COPY *.py /viasat/minitwit                                                                                                      0.0s
 => CACHED [builder 9/9] COPY *.sql /viasat/minitwit                                                                                                     0.0s
 => CACHED [tester 1/2] COPY requirements-dev.txt .                                                                                                      0.0s
 => CACHED [tester 2/2] RUN pip install -r requirements-dev.txt                                                                                          0.0s
 => exporting to oci image format                                                                                                                       20.4s
 => => exporting layers                                                                                                                                  0.0s
 => => exporting manifest sha256:ace8c0ce82a0983ccec690babe884bd918c7d6d968f745935e5946345971d63f                                                        0.0s
 => => exporting config sha256:7f00ff6e22fe07d274843f82c70f8bbbe4465db8e3474770772ce28f4763073c                                                          0.0s
 => => sending tarball                                                                                                                                  20.4s
 => importing to docker                                                                                                                                  0.4s
[+] Running 1/1
 ⠿ Container minitwit-minitwit-test-1  Recreated                                                                                                         0.3s
Attaching to minitwit-minitwit-test-1
minitwit-minitwit-test-1  | ============================= test session starts ==============================
minitwit-minitwit-test-1  | platform linux -- Python 3.8.16, pytest-5.3.2, py-1.11.0, pluggy-0.13.1
minitwit-minitwit-test-1  | rootdir: /viasat/minitwit
minitwit-minitwit-test-1  | collected 4 items
minitwit-minitwit-test-1  |
minitwit-minitwit-test-1  | test_minitwit.py ....                                                    [100%]
minitwit-minitwit-test-1  |
minitwit-minitwit-test-1  | =============================== warnings summary ===============================
minitwit-minitwit-test-1  | test_minitwit.py::test_register
minitwit-minitwit-test-1  |   /viasat/minitwit/minitwit.py:200: RemovedIn20Warning: Deprecated API features detected! These feature(s) are not compatible with SQLAlchemy 2.0. To prevent incompatible upgrades prior to updating applications, ensure requirements files are pinned to "sqlalchemy<2.0". Set environment variable SQLALCHEMY_WARN_20=1 to show all deprecation warnings.  Set environment variable SQLALCHEMY_SILENCE_UBER_WARNING=1 to silence this message. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)
minitwit-minitwit-test-1  |     the_db.execute(query.strip() + ';')
minitwit-minitwit-test-1  |
minitwit-minitwit-test-1  | -- Docs: https://docs.pytest.org/en/latest/warnings.html
minitwit-minitwit-test-1  | ======================== 4 passed, 1 warning in 36.06s =========================
minitwit-minitwit-test-1 exited with code 0

🔊 Run server

  • The runtime image only contains the needed binaries and python code to execute the server
    • That is, it does NOT carry the development sources
    • It only includes what it needs to execute
$ docker compose up minitwit-runtime
WARN[0000] Found orphan containers ([kind_easley amazing_murdock festive_almeida infallible_cannon recursing_torvalds fervent_almeida minitwit-minitwit-1]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
[+] Running 1/1
 ⠿ Container minitwit-minitwit-runtime-1  Recreated                                                                                                      0.5s
Attaching to minitwit-minitwit-runtime-1
minitwit-minitwit-runtime-1  | [2023-03-08 01:47:30,409] INFO in minitwit: Using local db sqlite:////var/minitwit/minitwit.db
minitwit-minitwit-runtime-1  |  * Serving Flask app 'minitwit.py'
minitwit-minitwit-runtime-1  |  * Debug mode: off
minitwit-minitwit-runtime-1  | WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
minitwit-minitwit-runtime-1  |  * Running on all addresses (0.0.0.0)
minitwit-minitwit-runtime-1  |  * Running on http://127.0.0.1:5000
minitwit-minitwit-runtime-1  |  * Running on http://192.168.192.2:5000
minitwit-minitwit-runtime-1  | Press CTRL+C to quit
  • It successfully creates the db file on demand outside the docker container.
☁️  [email protected]
☸️  [email protected] 📛 [email protected] 🎡 [email protected]    🐳 [email protected] 🐙 [email protected]
👮 marcellodesales
🏗  1.24.3+k3s1 🔐 rancher-desktop 🍱 default
~/dev/github.com/marcellodesales/minitwit on  feature/improve-experience-with-docker! 📅 03-07-2023 ⌚18:07:30
$ tree db
db
└── minitwit.db

0 directories, 1 file

$ file db/minitwit.db
db/minitwit.db: SQLite 3.x database, last written using SQLite version 3040001, file counter 5, database pages 5, cookie 0x3, schema 4, UTF-8, version-valid-for 5
  • Testing it works as before

Screenshot 2023-03-07 at 12 54 10 PM

Because pytest is included into the runtime one.

* When deploying in runtime, just run

pip install -r requirements.txt

* When developing, also execute the test

pip install -r requirements-dev.txt
Just building in Alpine linux, which is the lightest
and most secure docker image for this.

* Original entrypoint "run-app" now creates the database
  if the file doesn't exist in the original dir.
* Creates a new container

$ docker build --platform linux/amd64 -t minitwit .
[+] Building 2.9s (12/12) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                     0.0s
 => => transferring dockerfile: 37B                                                                                                                      0.0s
 => [internal] load .dockerignore                                                                                                                        0.0s
 => => transferring context: 2B                                                                                                                          0.0s
 => [internal] load metadata for docker.io/library/python:3.8.16-alpine3.17                                                                              0.5s
 => [1/7] FROM docker.io/library/python:3.8.16-alpine3.17@sha256:8518dd6657131d938f283ea97385b1db6724e35d45ddab6cd1c583796e35566a                        0.0s
 => [internal] load build context                                                                                                                        0.0s
 => => transferring context: 2.72kB                                                                                                                      0.0s
 => CACHED [2/7] RUN apk add musl-dev python3-dev mariadb-dev gcc build-base                                                                             0.0s
 => CACHED [3/7] WORKDIR /viasat/minitwit                                                                                                                0.0s
 => CACHED [4/7] COPY requirements.txt .                                                                                                                 0.0s
 => CACHED [5/7] RUN pip install -r requirements.txt                                                                                                     0.0s
 => [6/7] COPY . /viasat/minitwit                                                                                                                        0.1s
 => [7/7] RUN apk add bash                                                                                                                               2.1s
 => exporting to image                                                                                                                                   0.1s
 => => exporting layers                                                                                                                                  0.0s
 => => writing image sha256:e4578e2b7395797acf87fa395681357f9af127d1786f4c6329c3fd00281cef66                                                             0.0s
 => => naming to docker.io/library/minitwit                                                                                                              0.0s

* Running the app just create a new container
  * Specify a port to map to the opened one
  * Mount the volume so to save the state of the database

$ docker run -ti -v $(pwd)/db:/var/minitwit -p 4000:5000 minitwit
Initializing database at /var/minitwit/minitwit.db
[2023-03-07 20:53:04,424] INFO in minitwit: Using local db sqlite:////var/minitwit/minitwit.db
Initialized the database.
[2023-03-07 20:53:07,371] INFO in minitwit: Using local db sqlite:////var/minitwit/minitwit.db
 * Serving Flask app 'minitwit.py'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:5000
 * Running on http://172.17.0.2:5000
Press CTRL+C to quit
This is to help users reproduce the changes.
While executing the tests it may fail because the db doesn't exist
This is simpler to execute the build and run of the
container with the appropriate params.
When the database dir does not exist, the tests fail. Note that
the test cases are using the same database name as used by the
regular runtime environment...

The change fails the following errors in the test cases

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file

It's because the dir /var/minitwit does not exist before executing.
In a CI/CD environment, this would happen as the dir is expected.

From an empty container, here's the execution.

☁️  [email protected]
☸️  [email protected] 📛 [email protected] 🎡 [email protected]    🐳 [email protected] 🐙 [email protected]
👮 marcellodesales
🏗  1.24.3+k3s1 🔐 rancher-desktop 🍱 default
~/dev/github.com/marcellodesales/minitwit on  feature/improve-experience-with-docker! 📅 03-07-2023 ⌚16:41:53
$ docker run --platform linux/amd64 -ti -w $(pwd) -v $(pwd):$(pwd) --entrypoint bash viasat/minitwit
93078096ab1b:/Users/mdesales/dev/github.com/marcellodesales/minitwit# ^C
(failed reverse-i-search)`pip': ^C
93078096ab1b:/Users/mdesales/dev/github.com/marcellodesales/minitwit# pip install -r requirements-dev.txt
Collecting pytest==5.3.2
  Downloading pytest-5.3.2-py3-none-any.whl (234 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 234.5/234.5 KB 1.4 MB/s eta 0:00:00
Collecting pluggy<1.0,>=0.12
  Downloading pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
  Downloading wcwidth-0.2.6-py2.py3-none-any.whl (29 kB)
Collecting more-itertools>=4.0.0
  Downloading more_itertools-9.1.0-py3-none-any.whl (54 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.2/54.2 KB 2.5 MB/s eta 0:00:00
Collecting py>=1.5.0
  Downloading py-1.11.0-py2.py3-none-any.whl (98 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.7/98.7 KB 4.1 MB/s eta 0:00:00
Collecting attrs>=17.4.0
  Downloading attrs-22.2.0-py3-none-any.whl (60 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.0/60.0 KB 3.6 MB/s eta 0:00:00
Collecting packaging
  Downloading packaging-23.0-py3-none-any.whl (42 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.7/42.7 KB 2.7 MB/s eta 0:00:00
Installing collected packages: wcwidth, py, pluggy, packaging, more-itertools, attrs, pytest
Successfully installed attrs-22.2.0 more-itertools-9.1.0 packaging-23.0 pluggy-0.13.1 py-1.11.0 pytest-5.3.2 wcwidth-0.2.6
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 22.0.4; however, version 23.0.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
93078096ab1b:/Users/mdesales/dev/github.com/marcellodesales/minitwit# pytest test_minitwit.py
==================================================================== test session starts =====================================================================
platform linux -- Python 3.8.16, pytest-5.3.2, py-1.11.0, pluggy-0.13.1
rootdir: /Users/mdesales/dev/github.com/marcellodesales/minitwit
collected 4 items

test_minitwit.py E^C

=========================================================================== ERRORS ===========================================================================
______________________________________________________________ ERROR at setup of test_register _______________________________________________________________

self = Engine(sqlite:////var/minitwit/minitwit.db), fn = <bound method Pool.connect of <sqlalchemy.pool.impl.NullPool object at 0x4009b9c790>>
connection = None

    def _wrap_pool_connect(self, fn, connection):
        dialect = self.dialect
        try:
>           return fn()

/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:3361:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <sqlalchemy.pool.impl.NullPool object at 0x4009b9c790>

    def connect(self):
        """Return a DBAPI connection from the pool.

        The connection is instrumented such that when its
        ``close()`` method is called, the connection will be returned to
        the pool.

        """
>       return _ConnectionFairy._checkout(self)

/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py:327:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cls = <class 'sqlalchemy.pool.base._ConnectionFairy'>, pool = <sqlalchemy.pool.impl.NullPool object at 0x4009b9c790>, threadconns = None, fairy = None

    @classmethod
    def _checkout(cls, pool, threadconns=None, fairy=None):
        if not fairy:
>           fairy = _ConnectionRecord.checkout(pool)

/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py:894:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cls = <class 'sqlalchemy.pool.base._ConnectionRecord'>, pool = <sqlalchemy.pool.impl.NullPool object at 0x4009b9c790>

    @classmethod
    def checkout(cls, pool):
>       rec = pool._do_get()

/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py:493:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <sqlalchemy.pool.impl.NullPool object at 0x4009b9c790>

    def _do_get(self):
>       return self._create_connection()

/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py:256:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <sqlalchemy.pool.impl.NullPool object at 0x4009b9c790>

    def _create_connection(self):
        """Called by subclasses to create a new ConnectionRecord."""

>       return _ConnectionRecord(self)

/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py:273:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <sqlalchemy.pool.base._ConnectionRecord object at 0x4009eb8670>, pool = <sqlalchemy.pool.impl.NullPool object at 0x4009b9c790>, connect = True

    def __init__(self, pool, connect=True):
        self.__pool = pool
        if connect:
>           self.__connect()

/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py:388:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <sqlalchemy.pool.base._ConnectionRecord object at 0x4009eb8670>

    def __connect(self):
        pool = self.__pool

        # ensure any existing connection is removed, so that if
        # creator fails, this attribute stays None
        self.dbapi_connection = None
        try:
            self.starttime = time.time()
            self.dbapi_connection = connection = pool._invoke_creator(self)
            pool.logger.debug("Created new connection %r", connection)
            self.fresh = True
        except BaseException as e:
            with util.safe_reraise():
>               pool.logger.debug("Error on connect(): %s", e)

/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py:691:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <sqlalchemy.util.langhelpers.safe_reraise object at 0x4009eb86d0>, type_ = None, value = None, traceback = None

    def __exit__(self, type_, value, traceback):
        # see #2703 for notes
        if type_ is None:
            exc_type, exc_value, exc_tb = self._exc_info
            self._exc_info = None  # remove potential circular references
            if not self.warn_only:
>               compat.raise_(
                    exc_value,
                    with_traceback=exc_tb,
                )

/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py:70:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def raise_(
        exception, with_traceback=None, replace_context=None, from_=False
    ):
        r"""implement "raise" with cause support.

        :param exception: exception to raise
        :param with_traceback: will call exception.with_traceback()
        :param replace_context: an as-yet-unsupported feature.  This is
         an exception object which we are "replacing", e.g., it's our
         "cause" but we don't want it printed.    Basically just what
         ``__suppress_context__`` does but we don't want to suppress
         the enclosing context, if any.  So for now we make it the
         cause.
        :param from\_: the cause.  this actually sets the cause and doesn't
         hope to hide it someday.

        """
        if with_traceback is not None:
            exception = exception.with_traceback(with_traceback)

        if from_ is not False:
            exception.__cause__ = from_
        elif replace_context is not None:
            # no good solution here, we would like to have the exception
            # have only the context of replace_context.__context__ so that the
            # intermediary exception does not change, but we can't figure
            # that out.
            exception.__cause__ = replace_context

        try:
>           raise exception

/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py:211:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <sqlalchemy.pool.base._ConnectionRecord object at 0x4009eb8670>

    def __connect(self):
        pool = self.__pool

        # ensure any existing connection is removed, so that if
        # creator fails, this attribute stays None
        self.dbapi_connection = None
        try:
            self.starttime = time.time()
>           self.dbapi_connection = connection = pool._invoke_creator(self)

/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py:686:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

connection_record = <sqlalchemy.pool.base._ConnectionRecord object at 0x4009eb8670>

    def connect(connection_record=None):
        if dialect._has_events:
            for fn in dialect.dispatch.do_connect:
                connection = fn(dialect, connection_record, cargs, cparams)
                if connection is not None:
                    return connection
>       return dialect.connect(*cargs, **cparams)

/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/create.py:578:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <sqlalchemy.dialects.sqlite.pysqlite.SQLiteDialect_pysqlite object at 0x4009bb1040>, cargs = ('/var/minitwit/minitwit.db',), cparams = {}

    def connect(self, *cargs, **cparams):
        # inherits the docstring from interfaces.Dialect.connect
>       return self.dbapi.connect(*cargs, **cparams)
E       sqlite3.OperationalError: unable to open database file

/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py:598: OperationalError

The above exception was the direct cause of the following exception:

request = <SubRequest 'client' for <Function test_register>>

    @pytest.fixture
    def client(request):
        db_fd, minitwit.app.config['DATABASE'] = tempfile.mkstemp()
        minitwit.app.config['DATABASE'] = 'sqlite:///' + minitwit.app.config['DATABASE']
        client = minitwit.app.test_client()
        with minitwit.app.app_context():
>           minitwit.init_db()

test_minitwit.py:23:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
minitwit.py:190: in init_db
    the_db = get_db()
minitwit.py:174: in get_db
    g.db = DB_ENGINE.connect() #pylint: disable=assigning-non-slot
/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:3315: in connect
    return self._connection_cls(self, close_with_result=close_with_result)
/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:96: in __init__
    else engine.raw_connection()
/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:3394: in raw_connection
    return self._wrap_pool_connect(self.pool.connect, _connection)
/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:3364: in _wrap_pool_connect
    Connection._handle_dbapi_exception_noconnection(
/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:2198: in _handle_dbapi_exception_noconnection
    util.raise_(
/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py:211: in raise_
    raise exception
/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py:3361: in _wrap_pool_connect
    return fn()
/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py:327: in connect
    return _ConnectionFairy._checkout(self)
/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py:894: in _checkout
    fairy = _ConnectionRecord.checkout(pool)
/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py:493: in checkout
    rec = pool._do_get()
/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py:256: in _do_get
    return self._create_connection()
  1 # -*- coding: utf-8 -*-
  2 """
  3     MiniTwit Tests
  4     ~~~~~~~~~~~~~~
  5
  6     Tests the MiniTwit application.
  7
  8     :copyright: (c) 2015 by Armin Ronacher.
  9     :license: BSD, see LICENSE for more details.
 10 """
 11 import os
/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py:273: in _create_connection
    return _ConnectionRecord(self)
/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py:388: in __init__
    self.__connect()
/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py:691: in __connect
    pool.logger.debug("Error on connect(): %s", e)
/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py:70: in __exit__
    compat.raise_(
/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py:211: in raise_
    raise exception
/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py:686: in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/create.py:578: in connect
    return dialect.connect(*cargs, **cparams)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <sqlalchemy.dialects.sqlite.pysqlite.SQLiteDialect_pysqlite object at 0x4009bb1040>, cargs = ('/var/minitwit/minitwit.db',), cparams = {}

    def connect(self, *cargs, **cparams):
        # inherits the docstring from interfaces.Dialect.connect
>       return self.dbapi.connect(*cargs, **cparams)
E       sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
E       (Background on this error at: https://sqlalche.me/e/14/e3q8)

/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py:598: OperationalError
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
/usr/local/lib/python3.8/ast.py:227: KeyboardInterrupt
(to show a full traceback on KeyboardInterrupt use --full-trace)
====================================================================== 1 error in 6.88s ======================================================================
93078096ab1b:/Users/mdesales/dev/github.com/marcellodesales/minitwit# exit
exit

***** Here's with the patch

$ docker run --platform linux/amd64 -ti -w $(pwd) -v $(pwd):$(pwd) --entrypoint bash viasat/minitwit
602f533b1fa6:/Users/mdesales/dev/github.com/marcellodesales/minitwit# pip install -r requirements-dev.txt
Collecting pytest==5.3.2
  Downloading pytest-5.3.2-py3-none-any.whl (234 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 234.5/234.5 KB 1.4 MB/s eta 0:00:00
Collecting pluggy<1.0,>=0.12
  Downloading pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting packaging
  Downloading packaging-23.0-py3-none-any.whl (42 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.7/42.7 KB 3.5 MB/s eta 0:00:00
Collecting more-itertools>=4.0.0
  Downloading more_itertools-9.1.0-py3-none-any.whl (54 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.2/54.2 KB 3.2 MB/s eta 0:00:00
Collecting wcwidth
  Downloading wcwidth-0.2.6-py2.py3-none-any.whl (29 kB)
Collecting py>=1.5.0
  Downloading py-1.11.0-py2.py3-none-any.whl (98 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.7/98.7 KB 3.2 MB/s eta 0:00:00
Collecting attrs>=17.4.0
  Downloading attrs-22.2.0-py3-none-any.whl (60 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.0/60.0 KB 1.9 MB/s eta 0:00:00
Installing collected packages: wcwidth, py, pluggy, packaging, more-itertools, attrs, pytest
Successfully installed attrs-22.2.0 more-itertools-9.1.0 packaging-23.0 pluggy-0.13.1 py-1.11.0 pytest-5.3.2 wcwidth-0.2.6
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 22.0.4; however, version 23.0.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
(failed reverse-i-search)`pyte': pi^Cinstall -r requirements-dev.txt
602f533b1fa6:/Users/mdesales/dev/github.com/marcellodesales/minitwit# pytest test_minitwit.py
==================================================================== test session starts =====================================================================
platform linux -- Python 3.8.16, pytest-5.3.2, py-1.11.0, pluggy-0.13.1
rootdir: /Users/mdesales/dev/github.com/marcellodesales/minitwit
collected 4 items

test_minitwit.py ... .                                                                                                                                  [100%]

====================================================================== warnings summary ======================================================================
test_minitwit.py::test_register
  /Users/mdesales/dev/github.com/marcellodesales/minitwit/minitwit.py:200: RemovedIn20Warning: Deprecated API features detected! These feature(s) are not compatible with SQLAlchemy 2.0. To prevent incompatible upgrades prior to updating applications, ensure requirements files are pinned to "sqlalchemy<2.0". Set environment variable SQLALCHEMY_WARN_20=1 to show all deprecation warnings.  Set environment variable SQLALCHEMY_SILENCE_UBER_WARNING=1 to silence this message. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)
    the_db.execute(query.strip() + ';')

-- Docs: https://docs.pytest.org/en/latest/warnings.html
=============================================================== 4 passed, 1 warning in 29.50s ================================================================
602f533b1fa6:/Users/mdesales/dev/github.com/marcellodesales/minitwit# exit
This is the improvement to the Dockerfile to split tests and runtime.
See details on the README updated with instructions.
@marcellodesales marcellodesales force-pushed the feature/improve-experience-with-docker branch from 138327a to 549c259 Compare March 8, 2023 02:21
@marcellodesales marcellodesales force-pushed the feature/improve-experience-with-docker branch from 549c259 to b2a39f8 Compare March 8, 2023 02:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant