Add Migration NullReferenceException after EF Core 8 upgrade

[DizzyDeveloper]

We are currently upgrade from .net 6 to .net 8, and subsequently from ef core 6 to ef core 8.

After performing the upgrade I attempted to generate an arbitrary migration, however, the add migration call dotnet ef migrations add Test --context DatabaseContext --verbose fails.

I end up with a null reference exception as below:

System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.EntityFrameworkCore.Metadata.Internal.ColumnBase`1.Microsoft.EntityFrameworkCore.Metadata.IColumnBase.get_ProviderValueComparer()
   at Microsoft.EntityFrameworkCore.Migrations.Internal.MigrationsModelDiffer.Diff(IColumn source, IColumn target, DiffContext diffContext)+MoveNext()
   at Microsoft.EntityFrameworkCore.Migrations.Internal.MigrationsModelDiffer.DiffCollection[T](IEnumerable`1 sources, IEnumerable`1 targets, DiffContext diffContext, Func`4 diff, Func`3 add, Func`3 remove, Func`4[] predicates)+MoveNext() 
   at System.Linq.Enumerable.ConcatIterator`1.MoveNext()
   at Microsoft.EntityFrameworkCore.Migrations.Internal.MigrationsModelDiffer.Diff(ITable source, ITable target, DiffContext diffContext)+MoveNext()
   at Microsoft.EntityFrameworkCore.Migrations.Internal.MigrationsModelDiffer.DiffCollection[T](IEnumerable`1 sources, IEnumerable`1 targets, DiffContext diffContext, Func`4 diff, Func`3 add, Func`3 remove, Func`4[] predicates)+MoveNext() 
   at System.Linq.Enumerable.ConcatIterator`1.MoveNext()
   at Microsoft.EntityFrameworkCore.Migrations.Internal.MigrationsModelDiffer.Sort(IEnumerable`1 operations, DiffContext diffContext)
   at Microsoft.EntityFrameworkCore.Migrations.Internal.MigrationsModelDiffer.GetDifferences(IRelationalModel source, IRelationalModel target)
   at Microsoft.EntityFrameworkCore.Migrations.Design.MigrationsScaffolder.ScaffoldMigration(String migrationName, String rootNamespace, String subNamespace, String language)
   at Microsoft.EntityFrameworkCore.Design.Internal.MigrationsOperations.AddMigration(String name, String outputDir, String contextType, String namespace)
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.AddMigrationImpl(String name, String outputDir, String contextType, String namespace)
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.AddMigration.<>c__DisplayClass0_0.<.ctor>b__0()
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.<>c__DisplayClass3_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.Execute(Action action)

Furthermore, to try and reduce the size of the jump from ef core 6 to 8, I first upgrade to ef core 7, and ran the same add migration call as above. After this upgrade the migration ran happily, generating empty up/down steps, and updating the context snapshot , in a manner the appears appropriate.

After this upgrade (and leaving the generated context snapshot in place) I upgrade to ef core 8 (particularly 8.0.3), and ran the add migration call I mentioned above, and received the aforementioned null reference exception.

Additionally

If, however, I delete the context snapshot before running the add migration call, then the add migration call does succeed. But as you can expect the up/down steps wants to re-create the whole db. That is fine as I can just nuke those migration files, however, the snapshot comparison now shows changes that are concerning where it changes column types i.e. from int4range[] to int4multirange and text to character varying(13). Note there has been no changes in the models themselves.

But ignore those things the migration does work. However, this leads on to another issue when running some of our migration tests. Particularly I end up with this exception.

System.InvalidOperationException : GenerateNonNullSqlLiteral not supported on NpgsqlRowValueTypeMapping
   at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.Mapping.NpgsqlRowValueTypeMapping.GenerateNonNullSqlLiteral(Object value)
   at Microsoft.EntityFrameworkCore.Storage.RelationalTypeMapping.GenerateSqlLiteral(Object value)
   at Npgsql.EntityFrameworkCore.PostgreSQL.Migrations.NpgsqlMigrationsSqlGenerator.ColumnDefinition(String schema, String table, String name, ColumnOperation operation, IModel model, MigrationCommandListBuilder builder)....

But lets ignore that exception for now.

EF Core version: 8.0.3
Database provider: Npgsql.EntityFrameworkCore.PostgreSQL
Target framework: .NET 8.0
Operating system: Windows 10
IDE: Rider 2023.3.4

[roji]

I end up with a null reference exception as below:

As always, there's little we can do with just an exception stacktrace, and no code showing how it got generated... Are you able to submit a minimal, runnable code sample showing this happening?

Furthermore, to try and reduce the size of the jump from ef core 6 to 8, I first upgrade to ef core 7

I wouldn't recommend doing this - the total amount of work in moving two versions separately can very well be larger than going from 6 to 8.

from int4range[] to int4multirange

This is an expected, documented change - the provider supports the multirange types introduced in PostgreSQL 14, and by default maps range arrays to them. The odd thing is that this change was already done in EFCore.PG 6, so you should have already seen it.

text to character varying(13)

This doesn't sound familiar - please open an issue in the EFCore.PG repo with a minimal, runnable sample.

GenerateNonNullSqlLiteral not supported on NpgsqlRowValueTypeMapping

Similarly, please open an issue on the PG side with a runnable code sample.

[DizzyDeveloper]

Hi @roji,

Thanks for the reply, I hadn't originally put code samples as there is a quiet a bit of code, and I have no idea where this exception is coming from, i.e. which model or column is causing this issue. So I don't know what sample to give other than the whole project which I can't. I was kind of hoping that from the stack trace someone might have been able to point me at some general things that could be causing this. I know that is not straight forward.

I did the 6 -> 7 -> 8 to also confirm that the exception was being caused by 8 and not something introduced by 7. But from your comments I can happily shuffle things now to jump straight to 8.

As for the other issues, I will get around to them, want to see if solving this might solve the others consequently.

I am going to see if I can narrow it down further, and provide a sample if/when I can, but in the meantime, if you are aware of anything obvious in the behaviour in ef core 8 that might cause this issue based on where the exception is from to help me narrow things down would be much appreciated.

[roji]

@DizzyDeveloper thank you, maybe one of the other team members will have an idea (/cc @maumar, @ajcvickers).

[DizzyDeveloper]

Heya @roji
Managed to work it out. We have a custom NpgsqlTypeMappingSource NpgsqlInetWithMaskTypeMappingSource that is based on github issue 1158 from the efcore.og repo. Whether we still need it or not I am not sure. But when upgraded ef core 8 and the corresponding npgsql libraries the NpgsqlInetTypeMapping class changed to accept a clr type of either IPAddress or NpgsqlINet.
Originally I had updated it by passing in typeof(NpgsqlInet) i.e. new NpgsqlInetTypeMapping(typeof(NpgsqlINet)). However, when that was the case the tools when processing the corresponding column definition couldn't find the appropriate mapper leading to the null reference exception. Changing it to accept IPAddress fixed the issue i.e. new NpgsqlInetTypeMapping(typeof(IPAddress)).

Also sussed out the other issues where it was altering the column types, other than the GenerateNonNullSqlLiteral not supported on NpgsqlRowValueTypeMapping exception, I will raise an issue in the PG repo for that exception.

So I am closing up the issue as there isn't more on this at the moment. However, it would be nice if the ef core tools printed out the table and column that it was processing on when it encounters an exception, especially in this case. I attached a debugger to see what the column the tool was processing when it encounter the null reference exception.

[roji]

@DizzyDeveloper thanks for the info - makes sense.

However, it would be nice if the ef core tools printed out the table and column that it was processing on when it encounters an exception [...]

If I'm following correctly, the exception above is of the kind that is "not supposed to happen" (i.e. developer bug in provider code), only because you were changing an internal service (NpgsqlTypeMappingSource) - it's generally not really feasible to catch these and add information, unfortunately, since they can occur almost anywhere...

[Tinsulate]

It really would be nice to have more information about these NullReferenceException. I'm now fighting with mystic NullReferenceError when removing a migration. I'm new to ef migrations and it's really difficult to reason what might be wrong.