[Core feature] Event Based Sensors for bucket upload events

[ajbeach2]

Motivation: Why do you think this is important?

Processing files uploaded to GCS buckets and triggering processing based on the prefix is a common pattern. Different prefixs may trigger different workflows.

Right now, the FileSensor supports bucket keys, but not prefixes.

This use case is more generically a Pub/Sub sensor because typically you would configure the bucket to send a notification to a PubSub Topic based on the prefix.

Goal: What should the final outcome look like, ideally?

This feature would allow for triggering workflows from pub/sub topics. With this feature, it would be possible to setup workflows to be triggered by events from different prefixes in GCS buckets.

Describe alternatives you've considered

This use case can be satisfied by a custom Sensor class, or by having an external service that subscribes or polls events which triggers the workflows programmatically. These solutions are less convenient.

Propose: Link/Inline OR Additional context

The proposed solution is to create an PubSubSensor. There could also be an SQS or SNS sensor to support S3/AWS. A more generic class to support multiple topic/queue types my also be considered.

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[welcome]

Thank you for opening your first issue here! 🛠