Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Map Filter search by resource layer feature #10827

Merged
merged 76 commits into from
Sep 24, 2024
Merged
Show file tree
Hide file tree
Changes from 47 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
11d1b5c
includes geometries on feature props in getPopupData, re #10816
whatisgalen Apr 24, 2024
bc4bf3a
adds translation for map popup map-filter search, re #10816
whatisgalen Apr 24, 2024
a99651f
creates fn in map-filter to take in a feature, interrogate type, set …
whatisgalen Apr 24, 2024
1eebc61
adds link in popup footer to search near this feature, re #10816
whatisgalen Apr 24, 2024
074c511
passes in resourceid as arg to filterByFeatureGeom, re #10816
whatisgalen Apr 24, 2024
1d01240
takes in resourceid as arg in filterByFeatureGeom, re #10816
whatisgalen Apr 24, 2024
31ef333
rm for loop in geom.features, assumes 1 feature, re #10816
whatisgalen Apr 24, 2024
cf05766
simplify buffer assignment in filterByFeatureGeom, re #10816
whatisgalen Apr 24, 2024
54281be
makes filter feature geom on map static, un-editable, re #10816
whatisgalen Apr 24, 2024
0a2f425
alternates map-filter payload with featureid and resourceid to avoid …
whatisgalen Apr 24, 2024
5ab58d6
moves geoshape_query build logic into own method in map_filter, re #1…
whatisgalen Apr 24, 2024
1b1a573
imports other dsl builder methods into map_filter, re #10816
whatisgalen Apr 24, 2024
2922ce5
creates 2nd control flow for payload of featureid, resourceid into ma…
whatisgalen Apr 24, 2024
ea387b5
refactors standard spatial_query logic to use helper method create_ge…
whatisgalen Apr 24, 2024
f075998
creates alternate flow if no resourceid supplied to filterByGeom, sim…
whatisgalen Apr 25, 2024
1c4d836
moves duplicative code into helper method, alters search_query in pla…
whatisgalen Apr 25, 2024
f5181b1
refactors method signature to reflect inplace alteration, re #10816
whatisgalen Apr 25, 2024
2a9266d
writes test for spatial filter by featureid, resourceid, re #10816
whatisgalen Apr 27, 2024
4095957
ensure pre-assigned featureids not overwritten in datatype transform …
whatisgalen Apr 27, 2024
3c9878f
checks for geometries.length in map_popup before passing args, re #10816
whatisgalen Apr 27, 2024
805693e
minor cleanup of logic in filterByFeatureGeom, re #10816
whatisgalen Apr 27, 2024
dfb65e3
fixes polygon format in search_test, re #10816
whatisgalen Apr 28, 2024
cb3140d
minor tweak to search tests, re #10816
whatisgalen Apr 29, 2024
f9fac10
moves popup feature handling into map popup provider utils for easier…
whatisgalen Apr 30, 2024
37a5d45
refactors mapFilter filterByFeatureGeom, expecting map popup provider…
whatisgalen Apr 30, 2024
4dc361a
indent child comment ko blocks to improve readability, re #10816
whatisgalen Apr 30, 2024
57d6742
cleanup map popup provider method, re #10816
whatisgalen Apr 30, 2024
56b61de
minor optimize in featureid assignment, re #10787
whatisgalen May 2, 2024
dcbe236
changes transltn to match functionality descrip, re #10816
whatisgalen May 10, 2024
6afe8ca
migrates visibility logic of filter by feature to map popup provider,…
whatisgalen May 10, 2024
f8e4f48
include showFilterByFeature on feature props in map vm, re #10816
whatisgalen May 10, 2024
eab6138
rm unneeded draw.setup in mapfilter.clear, re #10816
whatisgalen May 10, 2024
9dda454
camelcase correction in method name sendFeatureToMapFilter, re #10816
whatisgalen May 10, 2024
a6de407
enforce consistent param name, re #10816
whatisgalen May 10, 2024
24a7299
accomodates single feature map query on restoreState, re #10816
whatisgalen May 10, 2024
1674617
enables feature geom editing if map popup feature not from resource-l…
whatisgalen May 10, 2024
820a911
grabs the correct feature based on id match in feature-based spatial …
whatisgalen May 11, 2024
387752c
move spatial feature tests into own module, re #10816
whatisgalen May 11, 2024
89e64d1
moves up mutate of search_results_obj earlier, adds buffered geom if …
whatisgalen May 11, 2024
ae5b220
includes commented implementation of turfjs polygon simplification, r…
whatisgalen May 11, 2024
2fb80c9
includes documentation for popup provider method, re #10816
whatisgalen May 11, 2024
bcab5d0
pass in idx of corresponding geom.feature to sendFeatureToMapFilter, …
whatisgalen May 13, 2024
91b1e98
rm unused commented code, move to docs, re #10816
whatisgalen May 13, 2024
8868cbd
selects geometries.feature corresponding to the feature clicked in po…
whatisgalen May 13, 2024
fcbfcd8
change default buffer behavior in filterByFeatureGeom, re #10816
whatisgalen May 13, 2024
cecdc2c
adds check for feature-filter geom match before access, query mod, re…
whatisgalen May 13, 2024
ddc814c
clarifies method docs in map-popup-provider, re #10816
whatisgalen May 13, 2024
6b67f99
rm refs to featureid, resourceid in filter by feature, re #10816
whatisgalen Jun 7, 2024
53918a1
alter trans text for filter by feature, re #10816
whatisgalen Jun 7, 2024
e13e233
merges conflict for latest linting styling, re #10816
whatisgalen Jun 11, 2024
f30d648
Merge branch 'archesproject-dev/7.6.x' into 10816_popup_map_filter
whatisgalen Jun 18, 2024
e1ea2f3
rm refs to popupFeatureObject index, re #10816
whatisgalen Jun 18, 2024
f8b0ba5
merge conflict from latest in dev/7.6.x
whatisgalen Aug 14, 2024
f18555c
find specific feature in multi-feature geom tiles, re #10816
whatisgalen Aug 15, 2024
52aad84
run black against map_filter.py
whatisgalen Aug 15, 2024
9782f7f
run black on spatial_search_tests.py
whatisgalen Aug 15, 2024
5f6717d
fix merge conflict with latest, clean up logic, re #10816
whatisgalen Aug 22, 2024
ede08dd
rm unneeded test for feature-less map-filter req, refactor get_respon…
whatisgalen Aug 22, 2024
ae1adfb
refactor get_response_json into utils, change kwarg names in search_t…
whatisgalen Aug 22, 2024
aa2fba0
clear filter geoms every time filterByFeatureGeom called, re #10816
whatisgalen Aug 22, 2024
84a16ac
fix merge conflict against latest dev/7.6.x
whatisgalen Aug 29, 2024
5672e47
restore setting feature_geom onto query_object.map_filter, re #10816
whatisgalen Aug 30, 2024
43a3e4f
Merge branch 'archesproject-dev/7.6.x' into 10816_popup_map_filter
whatisgalen Sep 18, 2024
dc23575
Update arches/app/media/js/utils/map-popup-provider.js
whatisgalen Sep 19, 2024
d85c4d2
Update arches/app/media/js/utils/map-popup-provider.js
whatisgalen Sep 19, 2024
dea8266
fix merge conflict, re #10816
whatisgalen Sep 20, 2024
3d84a0f
Merge branch '10816_popup_map_filter' of https://github.com/archespro…
whatisgalen Sep 20, 2024
0b735eb
update get_response_json helper method in tests, re #10816
whatisgalen Sep 20, 2024
43348ae
fix typo in const, add null check for foundFeature, re #10816
whatisgalen Sep 20, 2024
75acb85
update trans text for filterByFeature, re #10816
whatisgalen Sep 21, 2024
54d79d5
change filterByFeature icon to fa-search, re #10816
whatisgalen Sep 21, 2024
d0da3dd
duplicate featureid match find logic in the show method, re #10816
whatisgalen Sep 21, 2024
9b67cbc
add check for undefined featureid on popupfeature, refactor duplicate…
whatisgalen Sep 22, 2024
a90dfaa
fix "this" reference error, re #10816
apeters Sep 23, 2024
98e92dd
nit, re #10816
apeters Sep 23, 2024
3665ef0
only show filter by feature button on the seach page, re #10816
apeters Sep 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion arches/app/datatypes/datatypes.py
Original file line number Diff line number Diff line change
Expand Up @@ -998,7 +998,7 @@ def transform_value_for_tile(self, value, **kwargs):
geojson = json.loads(value)
if geojson["type"] == "FeatureCollection":
for feature in geojson["features"]:
feature["id"] = str(uuid.uuid4())
feature["id"] = feature.get("id", str(uuid.uuid4()))
arches_geojson = geojson
else:
raise TypeError
Expand Down
29 changes: 28 additions & 1 deletion arches/app/media/js/utils/map-popup-provider.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
define(['arches',
define([
'arches',
'knockout',
'templates/views/components/map-popup.htm'
], function(arches, ko, popupTemplate) {
Expand Down Expand Up @@ -40,6 +41,32 @@ define(['arches',
return features;
},

/**
* This method enables custom logic for how the feature in the popup should be handled and/or mutated en route to the mapFilter.
* @param popupFeatureObject - the javascript object of the feature and its associated contexts (e.g. mapCard).
* @param featureGeomIndex - int index corresponding to the feature that was clicked, used for lookup against an arry of geoms
* @required @method mapCard.filterByFeatureGeom()
* @required @send argument: @param feature - a geojson feature object
* with an @optional @property: feature.id corresponding to a uuid for that tile_geometrycollection_feature
* @optional @send argument: @param resourceid
*/
sendFeatureToMapFilter: function(popupFeatureObject, featureGeomIndex)
{
let feature = popupFeatureObject.geometries()[featureGeomIndex].geom.features[0];
popupFeatureObject.mapCard.filterByFeatureGeom(feature, popupFeatureObject.resourceinstanceid);
},

/**
* Determines whether to show the button for Filter By Feature
* @param popupFeatureObject - the javascript object of the feature and its associated contexts (e.g. mapCard).
* @param featureGeomIndex - int index corresponding to the feature that was clicked, used for lookup against an arry of geoms
* @returns {boolean} - whether to show "Filter by Feature" on map popup
* typically dependent on at least 1 feature with a geometry and/or a featureid/resourceid combo
*/
showFilterByFeature: function(popupFeatureObject, featureGeomIndex) {
return (ko.unwrap(popupFeatureObject.geometries) || []).length > 0;
},

};
return provider;
});
5 changes: 4 additions & 1 deletion arches/app/media/js/viewmodels/map.js
Original file line number Diff line number Diff line change
Expand Up @@ -344,14 +344,17 @@ define([
var data = feature.properties;
var id = data.resourceinstanceid;
data.showEditButton = self.canEdit;
const descriptionProperties = ['displayname', 'graph_name', 'map_popup'];
data.sendFeatureToMapFilter = mapPopupProvider.sendFeatureToMapFilter;
data.showFilterByFeature = mapPopupProvider.showFilterByFeature;
const descriptionProperties = ['displayname', 'graph_name', 'map_popup', 'geometries'];
if (id) {
if (!self.resourceLookup[id]){
data = _.defaults(data, {
'loading': true,
'displayname': '',
'graph_name': '',
'map_popup': '',
'geometries': [],
'feature': feature,
});
if (data.permissions) {
Expand Down
53 changes: 51 additions & 2 deletions arches/app/media/js/views/components/search/map-filter.js
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ define([
};

this.searchGeometries = ko.observableArray(null);
this.searchGeometryFeature = ko.observable(null);
this.searchAggregations = ko.observable();
this.selectedTool = ko.observable();
this.geoJSONString = ko.observable(undefined);
Expand Down Expand Up @@ -274,6 +275,35 @@ define([
}
}, this);

this.filterByFeatureGeom = function(feature, resourceid=null) {
if (feature.geometry.type == 'Point' && this.buffer() == 0) { this.buffer(25); }
if (feature.id && resourceid) {
self.searchGeometryFeature({
"featureid": feature.id,
"resourceid": resourceid,
"buffer": {
"width": this.buffer(),
"unit": this.bufferUnit()
},
"inverted": this.filter.inverted()
});
}
let currentSearchGeoms = self.searchGeometries();
this.draw.set({
bferguso marked this conversation as resolved.
Show resolved Hide resolved
"type": "FeatureCollection",
"features": [feature]
});
if (currentSearchGeoms != null) {
currentSearchGeoms.push(feature);
} else {
currentSearchGeoms = [feature];
}
self.searchGeometries(currentSearchGeoms);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be stored in this.filter.feature_collection? Storing the feature in a different variable is causing an exception on line 545: if (this.filter.feature_collection().features.length > 0) {: Uncaught (in promise) TypeError: can't access property "length", mapQuery.features is undefined

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the popup feature lacks a resourceid, its geometry does get stored in this.filter.feature_collection here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a change in my latest commits so that restoreState will work as expected, drawing a buffered polygon on the map

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm finding that if a feature geometry filter is applied, the search is refreshed, and then the Map Filter Enabled search criteria is removed, the filter isn't removed from the URL and the filter remains applied.

image

Adding another Filter by Feature then shows a Request Failed banner, and continues to add geometries to the map filter.

image

self.updateFilter();
// if feature is not from a resource-layer, enable feature editing in map
if (!!self.searchGeometryFeature()) { self.draw.changeMode('static'); }
};

var updateSearchResultPointLayer = function() {
var pointSource = self.map().getSource('search-results-points');
var agg = ko.unwrap(self.searchAggregations);
Expand Down Expand Up @@ -518,7 +548,11 @@ define([
this.getFilter('term-filter').addTag('Map Filter Enabled', this.name, this.filter.inverted);
}
this.filter.feature_collection().features[0].properties['inverted'] = this.filter.inverted();
queryObj[componentName] = ko.toJSON(this.filter.feature_collection());
if (!!this.searchGeometryFeature()) {
queryObj[componentName] = ko.toJSON(this.searchGeometryFeature());
} else {
queryObj[componentName] = ko.toJSON(this.filter.feature_collection());
}
} else {
delete queryObj[componentName];
}
Expand All @@ -533,7 +567,21 @@ define([
var hasSpatialFilter = false;
if (componentName in query) {
var mapQuery = JSON.parse(query[componentName]);
if (mapQuery.features.length > 0) {
if (mapQuery.featureid && mapQuery.resourceid) {
buffer = mapQuery.buffer;
bufferUnit = mapQuery.bufferUnit;
inverted = mapQuery.inverted;
this.searchGeometryFeature({
"featureid": mapQuery.id,
"resourceid": mapQuery.resourceid,
"buffer": {
"width": buffer,
"unit": bufferUnit
},
"inverted": inverted
});
hasSpatialFilter = true;
} else if (mapQuery.features.length > 0) {
hasSpatialFilter = true;
var properties = mapQuery.features[0].properties;
inverted = properties.inverted;
Expand Down Expand Up @@ -573,6 +621,7 @@ define([
},

clear: function(reset_features) {
this.searchGeometryFeature(null);
this.filter.feature_collection({
"type": "FeatureCollection",
"features": []
Expand Down
115 changes: 81 additions & 34 deletions arches/app/search/components/map_filter.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,10 @@
from django.utils.translation import gettext as _
from arches.app.models.system_settings import settings
from arches.app.utils.betterJSONSerializer import JSONSerializer, JSONDeserializer
from arches.app.search.elasticsearch_dsl_builder import Bool, Nested, Terms, GeoShape
from arches.app.search.elasticsearch_dsl_builder import Bool, Match, Query, Nested, Term, Terms, GeoShape
from arches.app.search.components.base import BaseSearchFilter
from arches.app.search.search_engine_factory import SearchEngineFactory
from arches.app.search.mappings import RESOURCES_INDEX

logger = logging.getLogger(__name__)

Expand All @@ -28,46 +30,57 @@ def append_dsl(self, search_results_object, permitted_nodegroups, include_provis
search_query = Bool()
querysting_params = self.request.GET.get(details["componentname"], "")
spatial_filter = JSONDeserializer().deserialize(querysting_params)
if details["componentname"] not in search_results_object:
search_results_object[details["componentname"]] = {}

if "features" in spatial_filter:
if len(spatial_filter["features"]) > 0:
feature_geom = spatial_filter["features"][0]["geometry"]
feature_properties = {}
if "properties" in spatial_filter["features"][0]:
feature_properties = spatial_filter["features"][0]["properties"]
buffer = {"width": 0, "unit": "ft"}
if "buffer" in feature_properties:
buffer = feature_properties["buffer"]
search_buffer = _buffer(feature_geom, buffer["width"], buffer["unit"])
feature_geom = JSONDeserializer().deserialize(search_buffer.geojson)
geoshape = GeoShape(
field="geometries.geom.features.geometry", type=feature_geom["type"], coordinates=feature_geom["coordinates"]
)

invert_spatial_search = False
if "inverted" in feature_properties:
invert_spatial_search = feature_properties["inverted"]

spatial_query = Bool()
if invert_spatial_search is True:
spatial_query.must_not(geoshape)
else:
spatial_query.filter(geoshape)

# get the nodegroup_ids that the user has permission to search
spatial_query.filter(Terms(field="geometries.nodegroup_id", terms=permitted_nodegroups))

if include_provisional is False:
spatial_query.filter(Terms(field="geometries.provisional", terms=["false"]))

elif include_provisional == "only provisional":
spatial_query.filter(Terms(field="geometries.provisional", terms=["true"]))

search_query.filter(Nested(path="geometries", query=spatial_query))

search_results_object["query"].add_query(search_query)

if details["componentname"] not in search_results_object:
search_results_object[details["componentname"]] = {}
add_geoshape_query_to_search_query(feature_geom, feature_properties, permitted_nodegroups, include_provisional, search_query)
search_results_object["query"].add_query(search_query)

elif "featureid" in spatial_filter and "resourceid" in spatial_filter:
se = SearchEngineFactory().create()
main_query = Query(se)
nested_query = Nested(path="geometries")
match_feature = Match(field="geometries.geom.features.id", query=spatial_filter["featureid"])

# Create a Bool query for conditions inside the nested path
bool_nested_query = Bool()
bool_nested_query.must(match_feature.dsl)
nested_query.add_query(bool_nested_query.dsl)

bool_query = Bool()
match_resource = Term(field="resourceinstanceid", term=spatial_filter["resourceid"])
bool_query.must(match_resource.dsl) # Match resource instance ID at the document level
bool_query.must(nested_query.dsl) # Add the nested query

# Set the entire bool query to the main query object
main_query.add_query(bool_query.dsl)

response = main_query.search(index=RESOURCES_INDEX)
geometries = []
for hit in response['hits']['hits']:
if len(geometries) > 0:
break
for geom in hit['_source']['geometries']:
if len(geometries) > 0:
break
for feature in geom['geom']['features']:
if len(geometries) > 0:
break
if feature['id'] == spatial_filter["featureid"]:
geometries.append(feature)

if len(geometries) > 0:
feature_geom = geometries[0]["geometry"]
buffered_feature_geom = add_geoshape_query_to_search_query(feature_geom, spatial_filter, permitted_nodegroups, include_provisional, search_query)
search_results_object[details["componentname"]] = buffered_feature_geom
search_results_object["query"].add_query(search_query)

try:
search_results_object[details["componentname"]]["search_buffer"] = feature_geom
Expand Down Expand Up @@ -98,3 +111,37 @@ def _buffer(geojson, width=0, unit="ft"):
res = cursor.fetchone()
geom = GEOSGeometry(res[0], srid=4326)
return geom

def add_geoshape_query_to_search_query(feature_geom, feature_properties, permitted_nodegroups, include_provisional, search_query):

buffer = {"width": 0, "unit": "ft"}
if "buffer" in feature_properties:
buffer = feature_properties["buffer"]
# feature_geom = spatial_filter["features"][0]["geometry"]
search_buffer = _buffer(feature_geom, int(buffer["width"]), buffer["unit"])
feature_geom = JSONDeserializer().deserialize(search_buffer.geojson)
geoshape = GeoShape(
field="geometries.geom.features.geometry", type=feature_geom["type"], coordinates=feature_geom["coordinates"]
)
invert_spatial_search = False
if "inverted" in feature_properties:
invert_spatial_search = feature_properties["inverted"]

spatial_query = Bool()
if invert_spatial_search is True:
spatial_query.must_not(geoshape)
else:
spatial_query.filter(geoshape)

# get the nodegroup_ids that the user has permission to search
spatial_query.filter(Terms(field="geometries.nodegroup_id", terms=permitted_nodegroups))

if include_provisional is False:
spatial_query.filter(Terms(field="geometries.provisional", terms=["false"]))

elif include_provisional == "only provisional":
spatial_query.filter(Terms(field="geometries.provisional", terms=["true"]))

search_query.filter(Nested(path="geometries", query=spatial_query))

return feature_geom
1 change: 1 addition & 0 deletions arches/app/templates/javascript.htm
Original file line number Diff line number Diff line change
Expand Up @@ -654,6 +654,7 @@
map-add-line='{% trans "Add line" as mapAddLine %} "{{ mapAddLine|escapejs }}"'
map-add-polygon='{% trans "Add polygon" as mapAddPolygon %} "{{ mapAddPolygon|escapejs }}"'
map-select-drawing='{% trans "Select drawing" as mapSelectDrawing %} "{{ mapSelectDrawing|escapejs }}"'
filter-by-feature='{% trans "Filter by Feature" as filterByFeature %} "{{ filterByFeature|escapejs }}"'
related-instance-map-sources='{% trans "Related instance map sources" as relatedInstanceMapSources %} "{{ relatedInstanceMapSources|escapejs }}"'
related-instance-map-source-layers='{% trans "Related instance map source layers (optional)" as relatedInstanceMapSourceLayers %} "{{ relatedInstanceMapSourceLayers|escapejs }}"'
intersection-layer-configuration='{% trans "Intersection layer configuration" as intersectionLayerConfiguration %} "{{ intersectionLayerConfiguration|escapejs }}"'
Expand Down
Loading