10 tests failing out 65

[cirosantilli]

On 8bc3a09 after the interactive browser usage seems OK, I tried:

MONGODB_FEATHERS_REALWORLD=mongodb://localhost:27017/mydb npm run test

and the failures were:

  55 passing (15s)
  10 failing

  1) 'articles' service - client
       Run tests using client and server
         creates a user, encrypts password, logs in:
     Conflict: username: value already exists.
      at new Conflict (node_modules/@feathersjs/errors/lib/index.js:142:17)
      at convert (node_modules/@feathersjs/errors/lib/index.js:239:7)
      at toError (node_modules/@feathersjs/rest-client/lib/base.js:12:9)
      at processTicksAndRejections (internal/process/task_queues.js:97:5)
      at async Context.<anonymous> (test/services/client-articles.test.js:54:14)

  2) 'tags' service
       Run tests using client and server
         creates a user, encrypts password, logs in:
     Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/ciro/git/feathers-realworld-example-app/test/services/tags.test.js)
      at listOnTimeout (internal/timers.js:549:17)
      at processTimers (internal/timers.js:492:7)

  3) 'tags' service
       Run tests using client and server
         Creates an Article with tags:

      AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert.ok(tags.tags.length === 2)

      + expected - actual

      -false
      +true

      at Context.<anonymous> (test/services/tags.test.js:67:14)
      at processTicksAndRejections (internal/process/task_queues.js:97:5)

  4) 'tags' service
       Run tests using client and server
         Adds another article checks for the tag order:

      AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert.ok(tags.tags.length === 3)

      + expected - actual

      -false
      +true

      at Context.<anonymous> (test/services/tags.test.js:79:14)
      at processTicksAndRejections (internal/process/task_queues.js:97:5)

  5) 'tags' service
       Run tests using client and server
         Adds a third article checks for the tag order:

      AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert.ok(tags.tags.length === 3)

      + expected - actual

      -false
      +true

      at Context.<anonymous> (test/services/tags.test.js:92:14)
      at processTicksAndRejections (internal/process/task_queues.js:97:5)

  6) 'tags' service
       Run tests using client and server
         Adds a fourth article checks for the tag order:

      AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert.ok(tags.tags.length === 3)

      + expected - actual

      -false
      +true

      at Context.<anonymous> (test/services/tags.test.js:105:14)
      at processTicksAndRejections (internal/process/task_queues.js:97:5)

  7) 'tags' service
       Run tests using client and server
         cleans up first article and checks tags:

      AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert.ok(tags.tags.length === 2)

      + expected - actual

      -false
      +true

      at Context.<anonymous> (test/services/tags.test.js:116:14)
      at processTicksAndRejections (internal/process/task_queues.js:97:5)

  8) 'tags' service
       Run tests using client and server
         cleans up another article and checks tags:

      AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert.ok(tags.tags.length === 2)

      + expected - actual

      -false
      +true

      at Context.<anonymous> (test/services/tags.test.js:127:14)
      at processTicksAndRejections (internal/process/task_queues.js:97:5)

  9) 'tags' service
       Run tests using client and server
         cleans up third article and checks tags:

      AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert.ok(tags.tags.length === 1)

      + expected - actual

      -false
      +true

      at Context.<anonymous> (test/services/tags.test.js:138:14)
      at processTicksAndRejections (internal/process/task_queues.js:97:5)

  10) 'tags' service
       Run tests using client and server
         cleans up fourth article and checks tags and cleans up user:

      AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert.ok(tags.tags.length === 0)

      + expected - actual

      -false
      +true

      at Context.<anonymous> (test/services/tags.test.js:178:14)
      at processTicksAndRejections (internal/process/task_queues.js:97:5)

Ubuntu 20.10.

1)

If I narrow that error to a single test run with manually cleaned DB:

MONGODB_FEATHERS_REALWORLD=mongodb://localhost:27017/mydb mocha test/services/client-articles.test.js --exit -g 'creates a user, encrypts password, logs in'
info: mongodb://localhost:27017/mydb2

the first failure is always:

  'articles' service - client
Feathers application started on http://localhost:3000
    Run tests using client and server
      1) creates a user, encrypts password, logs in


  0 passing (2s)
  1 failing

  1) 'articles' service - client
       Run tests using client and server
         creates a user, encrypts password, logs in:
     Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/ciro/git/feathers-realworld-example-app/test/services/client-articles.test.js)
      at listOnTimeout (internal/timers.js:549:17)
      at processTimers (internal/timers.js:492:7)

If I force an increased timeout it passes on a clean DB:

MONGODB_FEATHERS_REALWORLD=mongodb://localhost:27017/mydb4 mocha test/services/client-articles.test.js --exit -g 'creates a user, encrypts password, logs in' --timeout 10000

If I re-run on the same non-clean DB, I get the user already exists. So we just have to clean up the database before running those tests.

There's some documentation for this at: https://docs.feathersjs.com/guides/basics/testing.html#test-database-setup Their approach seems to be to just clear up the database once before the run, and then create users with different names for each test, which is not ideal. Rails has ways to truncate their SQL DBs: https://stackoverflow.com/questions/41911906/how-to-clean-database-before-running-the-each-spec-file we could just add a db.collection.drop()..

Another thing they are doing is:

      try {
        await app.service('users').create(userInfo);
      } catch (error) {
        // Do nothing, it just means the user already exists and can be tested
      }

So ugly.