Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvement of Project Creation with extended INTERLIS Models #61

Merged
merged 35 commits into from
Oct 3, 2023

Conversation

signedav
Copy link
Member

@signedav signedav commented Jul 14, 2023

Important note (esp. for documenters)

Conceptional thoughts

Assumptions

Since it appears almost impossible to care for all the cases, I need to make some assumptions what mostly would be the case.

  • When you extend a base class with the same name, you intend to "replace" it, otherwise you would rename it.
  • When you extend a base class multiple times (what you do with different names) then you intend to "replace" it.
  • Exception for the two cases above: When you extended the class it in the same model but another topic (because if you intent to "replace" it, you would have made it ABSTRACT).

Leads to

  • Base classes with same named extensions are irrelevant
  • Base classes with multiple extensions are irrelevant
  • Except if the extension is in the same model, then it's not irrelevant but will be renamed

And this means

Table is irrelevant when:

We find the table as baseClass with same named extension in another model OR we find the table in the baseClass with multiple extensions in another topic but the same or the extended model.

Smartness

This implementation concerns mostly smart 2 inheritance strategy. With smart 1 there could be the optimization that the relevant layer is set as default t_type, but this is not scope of this PR.

Limitations

There will be issues with the optimization. For example:

  • When a class is extended in another model (but not the one you import, but still a depending model, so it's imported as well) it could be hidden because of this, but still you would have needed it.
  • When you extend a topic and have a not-extended base class referencing to a base class that has an extension, then the referenced base class will be hidden. But you might want it.

What to do then? You have always the option for the NONE-strategy :-)

Backwards compatibility

Will there be issues with backwards compatibility since we do in every case a layer rename in case of multiple same named classes? - Yes, but this would only concern cases, where you already had troubles because of same named layers (e.g. UsabILIty Hub or get_feature-expressions.

Deliverables

No ambiguous layers anymore

Independently from extended models (but pretty much connected to it), we eliminate ambiguous layer naming in general. This is done in two steps:

  1. If layername is ambiuous append topic name as suffix.
  2. If layername is still ambiguous append model name as suffix.

In case we use the OptimizeStrategy.HIDE we don't rename ambiguous layers, when they will be hidden later on anyway.

With the Example Use Case Polymorphic below the result will look like this:

image

Emitting the "irrelevant" layers

The "irrelevant" layers are emitted in the backend even when not used according to the strategies influencing the layer tree, the relations and the basket fields.

The three strategies

You can set the strategy in the generator:

    class OptimizeStrategy(Enum):
        NONE = 0
        GROUP = 1
        HIDE = 2

Layer tree according to three optimize-stategies

Example NONE:

image

Example GROUP:

image

Example HIDE:

image

Here we see as well that on this strategy the renaming of ambiguous layers concerns if the layer will be hidden later (like Firma).

Relations according to two optimize-stategies (hide and rename/group)

Relations and widgets should be created or not (concerning the strategies):

NONE Strategy

Create relations of irrelevant layers and add relation editors to the form.

GROUP Strategy

Create relations of irrelevant layers BUT don't add relation editors to the form.

HIDE layer strategy

Don't create relation on irrelevant layers.

Basket Stretch Goal

Only the backend implementation could be done in this project. Here we get three information:

  • all_topics of layer: all the topics that this layer's class is located in. This can be used for NONE-strategy.
  • relevant_topics of layer: all the relevant topics, means the most extended ones (where users work in). This can be used for GROUP- and HIDE-strategy. (Btw. if it's empty it means the relevant topic is the one where the class is designed in.)
  • relevance of topics: If a topic is not extended anymore, then it's relevant. This information is provided but probably not used for the follow up implementation.

To have it fully integrated, this needs to be done: opengisch/QgisModelBaker#827

Relevance of topics

In the backend (QgisModelBakerLibrary) we detect the "relevance" of a topic. Means as long as it's not extended it is relevant, otherwise it's not (since it's extension is the relevant one). This is not used in frontend, but provides the possiblity for improvements (e.g. not creating irrelevant baskets.

Relevant topics in layer

This evaluates all the topics the layers are relevant. If it's not the topic the class is designed in (then this value would be empty), then it's a list of all the topics that are most extended (could be multiple).


Example Use Cases:

1. Ortsplanung

What is it about

More or less reallive example with 3 levels and multiple extensions. Level 2 extends level 1 and level 3 extends parts of level 2 and parts of level 1.

  • Ortsplanung_V1_1 using (not extending) Infrastruktur_V1.
  • Kantonale_Ortsplanung_V1_1 extending Ortsplanung_V1_1 (TOPIC Konstruktionen).
  • Gewerbe_V1.
  • Staedtisches_Gewerbe_V1 extending Gewerbe_V1.
  • Staedtische_Ortsplanung_V1_1 extending Kantonale_Otsplanung_V1_1 (TOPIC Konstruktionen) and using Staedtisches_Gewerbe_V1.

Goals

This a creation in the 3 versions. The renaming of the layer has been done manually.

image

  • Layertree: Only relevant layers should be displayed (or others renamed and grouped) Means in this example above:
    • on level 1 it stays
    • on level 2 Gebaeude (from base) should be hidden
    • on level 3 Gebaeude (from base) / Gebaeude kantnl_ng... / Firma (from base) should be hidden
    • on level 1 it stays
  • Relations: To do...

2. Polymorphic

What is it about

  • Polymorphic_Ortsplanung_V1_1 extending Ortsplanung_V1_1 (TOPIC Konstruktionen) multiple times.
  • Extending topics from the same model (TOPIC IndustrieGewerbe) and extending classes for multiple purposes (in TOPIC Hallen)
  • Another topic called Konstruktionen with a class Gebaeude that stands independently and is not an extension of anything.

Goals

(The renaming of the layer Gebaeude has been done manually)
image

  • Layertree: - Only relevant layers and proper renaming of ambiguous layers like from Freizeit.Gebaeude and Gewerbe.Gebaeude etc. Means in this example above:
    • Gebaeude (from base) should be hidden or grouped
    • Gebaeude polymrpgn_v1_1gewerbe_gebaeude or Gebaeude polymrpgn_v1_1hallen_gebaeude might be not used, but still we cannot hide it because we don't know. If it would be ABSTRACT, then it would be hidden already (not created by ili2db - like Turnhalle). We optimize per Model and not per Topic, so it stays.
    • Other Gebauede are ambiguous, so they should be renamed
  • Relations: To do

3. Associations and Structures

What is it about

  • Bauplanung_V1_1 using (not extending) Infrastruktur_V1 having STRUCTURE Material and a many-to-many ASSOCIATION to Strassen as well a TOPIC Natur with two associated classes.
  • Kantonale_Bauplanung_V1_1 extending Bauplanung_V1_1 (STRUCTURE and adding attribute in Class, so there is a link on base structure and another to extended structure) and the Natur TOPIC just to another attribute to the classes.

Goal

image

  • Layertree: Only relevant layers and proper renaming of ambiguous layers like from Material and Material etc.
    • Since Material from base model is still needed, so it should not be hidden but renamed (not sure if possible and really nice practice like it's modelled)
    • Strassen_Gebaeude (from base model) should be hidden
    • Park (from base model) should be hidden.
    • Bruttstelle (from base model) should be visible (since it's not extended)
    • Buntbrache (from base model) should be visible (since it's renamed)
    • Feld (from base model) should not be visible (since it's extended multiple times)
    • Should Tierart (from base model) be visible? Brutstelle references to it in the base model, but since it's extended, I'm not sure if it should still reference to it but to it's extension...
  • Relations: To do

Further informations

SQL to achieve that

-- We find the table as baseClass with same named extension in another model 
-- OR we find the table in the baseClass with multiple extensions in another 
-- but the same model.

-- used to get the class names from the full names
WITH names AS (
  WITH class_level_name AS(
	  WITH topic_level_name AS (
		  SELECT 
		  thisClass as fullname,
		  substr(thisClass, 0, instr(thisClass, '.')) as model,
		  substr(ltrim(thisClass,substr(thisClass, 0, instr(thisClass, '.'))),2) as topicclass
		  FROM T_ILI2DB_INHERITANCE
	  )
	  SELECT *, ltrim(topicclass,substr(topicclass, 0, instr(topicclass, '.'))) as class_with_dot
	  FROM topic_level_name
  ) select fullname, model, topicclass, substr(class_with_dot, instr(class_with_dot,'.')+1) as class
  FROM class_level_name
)
SELECT i.baseClass as base, i.thisClass as extension
FROM T_ILI2DB_INHERITANCE i
LEFT JOIN names extend_names
ON thisClass = extend_names.fullname
LEFT JOIN names base_names
ON baseClass = base_names.fullname
-- it's extended
WHERE baseClass IS NOT NULL
-- in a different model
AND base_names.model != extend_names.model
AND ( 
	-- with the same name
	base_names.class = extend_names.class
	OR 
	-- multiple times in the same extended model
	(SELECT COUNT(baseClass) FROM T_ILI2DB_INHERITANCE JOIN names extend_names ON thisClass = extend_names.fullname WHERE baseClass = i.baseClass GROUP BY baseClass, extend_names.model)>1
)

Examples:

1. Ortsplanung on level 3 (Staedtische...)

Gebaeude (from base) / Gebaeude kantnl_ng... / Firma (from base) should be hidden:

image

In this case we could just hide all the ones listet in baseClass

2. Polymorphic

Gebaeude polymrpgn_v1_1gewerbe_gebaeude or Gebaeude polymrpgn_v1_1hallen_gebaeude might be not used, but still we cannot hide it because we don't know. If it would be ABSTRACT, then it would be hidden already (not created by ili2db - like Turnhalle). We optimize per Model and not per Topic, so it stays.

image

In this case all the ones listed in baseClass (except the one being in the same model) could be hidden.

3. Associations and Structures

Since Material and Tierart from the base model is still needed, so it should not be hidden but renamed (but this is the special case), so we want to hide Park, Gebaude, Tierart, Material and the assoc Strassen_Gebaeude from the base model.

image

In this case all teh ones listed in baseClass being extended with the same classname could be hidden (Bauplanung_V1_1.Konstruktionen.Material, Bauplanung_V1_1.Konstruktionen.Gebaeude, Bauplanung_V1_1.Natur.Tierart, Bauplanung_V1_1.Natur.Park) and "Bauplanung_V1_1.Natur.Feld" since it has multiple extensions with another name in another model.

Special case

  • In Bauplanung-Model: Since Material from base model is still needed, so it should not be hidden but renamed - but here I am not sure if this is really bad practice to still use the baseClass but not having the extension renamed...
  • Should Tierart (from base model) be visible? Brutstelle references to it in the base model, but since it's extended, it would be hidden. I'm not sure if it's correct. When I extend a class, should the other baseClasses reference to the extension instead?
  • When a class is extended in another model (but not the one you import, but still a depending model, so it's imported as well) it could be hidden because of this, but still you would have needed it.

Extended approach (obsolete)

Obsolete but considering the special case

    1. Is this table part of the imported model (means the model we want to work with and most possibly the most extended one (but maybe still not the only most extended one))?
    • Yes: a. Table is relevant
    • No: Go on...
    1. Is this table NOT listed as baseClass in t_ili2db_inheritance (means there exists none extension of it)?
    • Yes: b. Table is relevant
    • No: Go on...
    1. Is this table contained in t_ili2db_column_prop.tablename (as a child) or in t_ili2db_column_prop.settings and t_ili2db_column_prop.tag is 'ch.ehi.ili2db.foreignKey' and the t_ili2db_column_prop.setting/ t_ili2db_column_prop.tablename contains a relevant table that is not extended?
    • Yes: Go on
    • No: Table is not relevant
    1. No extension of the table linking to this relevant table above as well?
    • Yes: Table is relevant
    • No: Table is not relevant
%%{init: {'flowchart' : {'curve' : 'linear'}}}%%
flowchart LR
    Q1{1. Part of the imported model?}
    Q1 --> |Yes|R[Table is relevant]
    Q1 --> |No|Q2{2. NOT listed as baseClass?}
    Q2 --> |Yes|R
    Q2 --> |No|Q3{3. Any relation for this table?}
    Q3 --> |Yes|Q4{4. No extension linking?}
    Q3 --> |No|R2[Table is NOT relevant]
    Q4 --> |Yes|R
    Q4 --> |No|R2
Loading

SQL to achieve that

Statement to check 3 and 4 of table fauna

WITH targets AS ( 
  -- statement wohin 'fauna' als child zeigt
  SELECT c.iliname as name
  FROM T_ILI2DB_COLUMN_PROP AS cprop
  LEFT JOIN t_ili2db_classname c
	  ON cprop.setting == c.sqlname
  WHERE cprop.tag = 'ch.ehi.ili2db.foreignKey' 
  AND cprop.tablename = 'fauna'
  -- nur wenn auf klasse des modells gelinkt wird
  AND substr(c.iliname, 0, instr(c.iliname, '.')) = 'Kantonale_Bauplanung_V1_1'
  AND ( -- berücksichtige nur classen, die auch extended sind (in baseClass vorhanden)
        SELECT COUNT(*) 
			  FROM T_ILI2DB_INHERITANCE
			  LEFT JOIN t_ili2db_classname c
			  ON 'fauna' == c.sqlname
			  WHERE baseClass = c.iliname
	  )
),
same_parents AS (
  -- zeigen wieviele vererbungen von 'fauna' auf dieselben objekte referenzieren wie fauna
	SELECT COUNT(*) AS count
	FROM targets
	INNER JOIN
	(
	        -- statement wohin vererbungen von 'fauna' als child zeigen
		SELECT parent.iliname as name
		FROM T_ILI2DB_COLUMN_PROP AS cprop
		LEFT JOIN t_ili2db_classname child
			ON cprop.tablename == child.sqlname
		LEFT JOIN t_ili2db_classname parent
			ON cprop.setting == parent.sqlname
		WHERE cprop.tag = 'ch.ehi.ili2db.foreignKey'
		AND substr(child.iliname, 0, instr(child.iliname, '.')) = 'Kantonale_Bauplanung_V1_1'
		AND child.iliname IN ( 
			SELECT thisClass
			FROM T_ILI2DB_INHERITANCE
			LEFT JOIN t_ili2db_classname c
			ON 'fauna' == c.sqlname
			WHERE baseClass = c.iliname
		)
	) AS other_parents
	ON targets.name = other_parents.name
),
sources AS ( 
  -- statement wovon 'fauna' als parent referenziert wird:
	SELECT c.iliname as name
  FROM T_ILI2DB_COLUMN_PROP AS cprop
  LEFT JOIN t_ili2db_classname c
	  ON cprop.tablename == c.sqlname
  WHERE cprop.tag = 'ch.ehi.ili2db.foreignKey' 
  AND cprop.setting = 'fauna'
  AND ( -- berücksichtige nur classen, die auch extended sind (in baseClass vorhanden)
        SELECT COUNT(*) 
			  FROM T_ILI2DB_INHERITANCE
			  LEFT JOIN t_ili2db_classname c
			  ON 'fauna' == c.sqlname
			  WHERE baseClass = c.iliname
	  )
),
same_children AS (
	SELECT COUNT(*) AS count
	FROM sources
	INNER JOIN
	(
    --statement wovon vererbungen von 'fauna' als parent referenziert werden:
	  SELECT child.iliname as name
    FROM T_ILI2DB_COLUMN_PROP AS cprop
    LEFT JOIN t_ili2db_classname child
	    ON cprop.tablename == child.sqlname
    LEFT JOIN t_ili2db_classname parent
	    ON cprop.setting == parent.sqlname
    WHERE cprop.tag = 'ch.ehi.ili2db.foreignKey'
    AND parent.iliname IN ( 
	    SELECT thisClass
	    FROM T_ILI2DB_INHERITANCE
	    LEFT JOIN t_ili2db_classname c
	    ON 'fauna' == c.sqlname
	    WHERE baseClass = c.iliname
    )
  ) AS other_children
  ON sources.name = other_children.name
)
SELECT TRUE 
WHERE (
  SELECT TRUE
  FROM targets
  JOIN same_parents
  -- if no targets found at the beginning, then it should be FALSE and if there are targets, but matches as well, it should be FALSE as well
  WHERE targets.name IS NOT NULL AND same_parents.count = 0
) OR (
  SELECT TRUE
  FROM sources
  JOIN same_children
  -- if no targets found at the beginning, then it should be FALSE and if there are targets, but matches as well, it should be FALSE as well
  WHERE sources.name IS NOT NULL AND same_children.count = 0
)

Old stuff (for archive)

-----------------------
FAUNA:

WITH targets AS ( 
  -- statement wohin 'fauna' als child zeigt
  SELECT c.iliname as name
  FROM T_ILI2DB_COLUMN_PROP AS cprop
  LEFT JOIN t_ili2db_classname c
	  ON cprop.setting == c.sqlname
  WHERE cprop.tag = 'ch.ehi.ili2db.foreignKey' 
  AND cprop.tablename = 'fauna'
  -- nur wenn auf klasse des modells gelinkt wird
  AND substr(c.iliname, 0, instr(c.iliname, '.')) = 'Kantonale_Bauplanung_V1_1'
  AND ( -- berücksichtige nur classen, die auch extended sind (in baseClass vorhanden)
        SELECT COUNT(*) 
			  FROM T_ILI2DB_INHERITANCE
			  LEFT JOIN t_ili2db_classname c
			  ON 'fauna' == c.sqlname
			  WHERE baseClass = c.iliname
	  )
),
same_parents AS (
  -- zeigen wieviele vererbungen von 'fauna' auf dieselben objekte referenzieren wie fauna
	SELECT COUNT(*) AS count
	FROM targets
	INNER JOIN
	(
	        -- statement wohin vererbungen von 'fauna' als child zeigen
		SELECT parent.iliname as name
		FROM T_ILI2DB_COLUMN_PROP AS cprop
		LEFT JOIN t_ili2db_classname child
			ON cprop.tablename == child.sqlname
		LEFT JOIN t_ili2db_classname parent
			ON cprop.setting == parent.sqlname
		WHERE cprop.tag = 'ch.ehi.ili2db.foreignKey'
		AND substr(child.iliname, 0, instr(child.iliname, '.')) = 'Kantonale_Bauplanung_V1_1'
		AND child.iliname IN ( 
			SELECT thisClass
			FROM T_ILI2DB_INHERITANCE
			LEFT JOIN t_ili2db_classname c
			ON 'fauna' == c.sqlname
			WHERE baseClass = c.iliname
		)
	) AS other_parents
	ON targets.name = other_parents.name
)
SELECT TRUE
FROM targets
JOIN same_parents
-- if no targets found at the beginning, then it should be FALSE and if there are targets, but matches as well, it should be FALSE as well
WHERE targets.name IS NOT NULL AND same_parents.count = 0

-------------------------------------------
PARK

-- zeigen wieviele vererbungen von 'park' von denselben objekten referenziert wird

WITH sources AS ( 
  -- statement wovon 'park' als parent referenziert wird:
	SELECT c.iliname as name
  FROM T_ILI2DB_COLUMN_PROP AS cprop
  LEFT JOIN t_ili2db_classname c
	  ON cprop.tablename == c.sqlname
  WHERE cprop.tag = 'ch.ehi.ili2db.foreignKey' 
  AND cprop.setting = 'park'
  AND ( -- berücksichtige nur classen, die auch extended sind (in baseClass vorhanden)
        SELECT COUNT(*) 
			  FROM T_ILI2DB_INHERITANCE
			  LEFT JOIN t_ili2db_classname c
			  ON 'park' == c.sqlname
			  WHERE baseClass = c.iliname
	  )
),
same_children AS (
	SELECT COUNT(*) AS count
	FROM sources
	INNER JOIN
	(
    --statement wovon vererbungen von 'park' als parent referenziert werden:
	  SELECT child.iliname as name
    FROM T_ILI2DB_COLUMN_PROP AS cprop
    LEFT JOIN t_ili2db_classname child
	    ON cprop.tablename == child.sqlname
    LEFT JOIN t_ili2db_classname parent
	    ON cprop.setting == parent.sqlname
    WHERE cprop.tag = 'ch.ehi.ili2db.foreignKey'
    AND parent.iliname IN ( 
	    SELECT thisClass
	    FROM T_ILI2DB_INHERITANCE
	    LEFT JOIN t_ili2db_classname c
	    ON 'park' == c.sqlname
	    WHERE baseClass = c.iliname
    )
  ) AS other_children
  ON sources.name = other_children.name
)
SELECT TRUE
FROM sources
JOIN same_children
-- if no targets found at the beginning, then it should be FALSE and if there are targets, but matches as well, it should be FALSE as well
WHERE sources.name IS NOT NULL AND same_children.count = 0

Examples:

1. Ortsplanung on level 3 (Staedtische...)

Gebaeude (from base) / Gebaeude kantnl_ng... / Firma (from base) should be hidden:

image

In this case we could just hide all the ones listet in baseClass

2. Polymorphic

Gebaeude polymrpgn_v1_1gewerbe_gebaeude or Gebaeude polymrpgn_v1_1hallen_gebaeude might be not used, but still we cannot hide it because we don't know. If it would be ABSTRACT, then it would be hidden already (not created by ili2db - like Turnhalle). We optimize per Model and not per Topic, so it stays.

image

In this case all the ones listed in baseClass (except the one being in the same model) could be hidden.

3. Associations and Structures

Since Material from base model is still needed, so it should not be hidden but renamed

image

Means we cannot hide all those from baseClass, when it's listed in t_ili2db_column_prop.tablename with an FK to a relevant table.

image

Other models

@signedav signedav changed the title testdata (ili-models) for the use cases Improvement of Project Creation with extended INTERLIS Models Jul 14, 2023
Copy link
Member

@domi4484 domi4484 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

Just as discussed I would put the base layers ad system groups into an another Extra group to keep the project even cleaner

modelbaker/dbconnector/mssql_connector.py Outdated Show resolved Hide resolved
modelbaker/generator/generator.py Outdated Show resolved Hide resolved
@signedav signedav requested a review from domi4484 October 1, 2023 20:39
@signedav signedav merged commit 4110201 into main Oct 3, 2023
4 checks passed
@signedav signedav deleted the extended-models branch October 3, 2023 08:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants