-
Notifications
You must be signed in to change notification settings - Fork 24
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* add tests to validate DAG yaml * up airflow to 2.8.1 --------- Co-authored-by: vitor <[email protected]>
- Loading branch information
1 parent
6327996
commit 70a45b3
Showing
22 changed files
with
294 additions
and
28 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,7 +2,7 @@ name: CI Tests | |
|
|
||
| on: | ||
| push: | ||
| branches: [ main ] | ||
| branches: [ "*" ] | ||
| pull_request: | ||
| branches: [ main ] | ||
|
|
||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| { | ||
| "yaml.schemas": { | ||
| // XXX para usar um schema do repo oficial https://github.com/gestaogovbr/Ro-dou: | ||
| // "https://raw.githubusercontent.com/gestaogovbr/Ro-dou/main/schemas/ro-dou.json": | ||
| // XXX para usar um schema de arquivo local: | ||
| "./schemas/ro-dou.json": [ | ||
| // XXX edit to your DAGs yaml directory | ||
| "dag_confs/**/*.yml", | ||
| "dag_confs/**/*.yaml" | ||
| ], | ||
| }, | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -16,5 +16,5 @@ dag: | |
| report: | ||
| emails: | ||
| - [email protected] | ||
| attach_csv: On | ||
| attach_csv: True | ||
| subject: "[String] com caracteres especiais deve estar entre aspas" | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -8,6 +8,6 @@ dag: | |
| report: | ||
| emails: | ||
| - [email protected] | ||
| attach_csv: On | ||
| attach_csv: True | ||
| subject: "Exemplo de termos na Variável" | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,202 @@ | ||
| { | ||
| "$schema": "http://json-schema.org/draft-07/schema#", | ||
| "type": "object", | ||
| "properties": { | ||
| "dag": { | ||
| "type": "object", | ||
| "description": "Instanciação da DAG", | ||
| "properties": { | ||
| "id": { | ||
| "type": "string", | ||
| "description": "Nome único da DAG" | ||
| }, | ||
| "description": { | ||
| "type": "string", | ||
| "description": "Descrição da DAG" | ||
| }, | ||
| "tags": { | ||
| "type": "array", | ||
| "description": "Lista de tags para filtragem da DAG no Airflow", | ||
| "items": { | ||
| "type": "string" | ||
| } | ||
| }, | ||
| "owner": { | ||
| "type": "array", | ||
| "description": "Lista de owners para filtragem da DAG no Airflow", | ||
| "items": { | ||
| "type": "string" | ||
| } | ||
| }, | ||
| "schedule": { | ||
| "type": "string", | ||
| "description": "Expressão cron válida ou uma das seguintes strings: @once, @continuous, @hourly, @daily, @weekly, @monthly, @quarterly, @yearly.", | ||
| "pattern": "^(\\S+\\s+){4}\\S+$|^@(once|continuous|hourly|daily|weekly|monthly|quarterly|yearly)$" | ||
| }, | ||
| "search": { | ||
| "type": "object", | ||
| "description": "Seção para definição da busca no Diário", | ||
| "properties": { | ||
| "sources": { | ||
| "type": "array", | ||
| "description": "description", | ||
| "items": { | ||
| "type": "string", | ||
| "enum": ["QD", "DOU"] | ||
| } | ||
| }, | ||
| "territory_id": { | ||
| "type": "integer", | ||
| "description": "Id do território no Querido Diário - QD" | ||
| }, | ||
| "terms": { | ||
| "oneOf": [ | ||
| { | ||
| "type": "array", | ||
| "description": "Lista de termos de busca no Diário", | ||
| "items": { | ||
| "type": "string" | ||
| } | ||
| }, | ||
| { | ||
| "type": "object", | ||
| "description": "description", | ||
| "properties": { | ||
| "from_airflow_variable": { | ||
| "type": "string", | ||
| "description": "description" | ||
| }, | ||
| "from_db_select": { | ||
| "type": "object", | ||
| "description": "description", | ||
| "properties": { | ||
| "sql": { | ||
| "type": "string", | ||
| "description": "description" | ||
| }, | ||
| "conn_id": { | ||
| "type": "string", | ||
| "description": "description" | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| ] | ||
| }, | ||
| "field": { | ||
| "type": "string", | ||
| "description": "description", | ||
| "enum": ["TUDO", "TITULO", "CONTEUDO"] | ||
| }, | ||
| "is_exact_search": { | ||
| "type": "boolean", | ||
| "description": "description" | ||
| }, | ||
| "ignore_signature_match": { | ||
| "type": "boolean", | ||
| "description": "description" | ||
| }, | ||
| "force_rematch": { | ||
| "type": "boolean", | ||
| "description": "description" | ||
| }, | ||
| "date": { | ||
| "type": "string", | ||
| "description": "description", | ||
| "enum": [ | ||
| "DIA", | ||
| "SEMANA", | ||
| "MES", | ||
| "ANO" | ||
| ] | ||
| }, | ||
| "dou_sections": { | ||
| "type": "array", | ||
| "description": "description", | ||
| "items": { | ||
| "type": "string", | ||
| "enum": [ | ||
| "SECAO_1", | ||
| "SECAO_2", | ||
| "SECAO_3", | ||
| "EDICAO_EXTRA", | ||
| "EDICAO_EXTRA_1A", | ||
| "EDICAO_EXTRA_1B", | ||
| "EDICAO_EXTRA_1D", | ||
| "EDICAO_EXTRA_2A", | ||
| "EDICAO_EXTRA_2B", | ||
| "EDICAO_EXTRA_2D", | ||
| "EDICAO_EXTRA_3A", | ||
| "EDICAO_EXTRA_3B", | ||
| "EDICAO_EXTRA_3D", | ||
| "EDICAO_SUPLEMENTAR", | ||
| "TODOS" | ||
| ] | ||
| } | ||
| } | ||
| }, | ||
| "required": ["terms"], | ||
| "additionalProperties": false | ||
| }, | ||
| "doc_md": { | ||
| "type": "string", | ||
| "description": "description" | ||
| }, | ||
| "report": { | ||
| "type": "object", | ||
| "description": "Aceita: `slack`, `discord`, `emails`, `attach_csv`, `subject`, `skip_null`", | ||
| "properties": { | ||
| "slack": { | ||
| "type": "object", | ||
| "description": "description", | ||
| "properties": { | ||
| "webhook": { | ||
| "type": "string", | ||
| "description": "description", | ||
| "format": "uri-reference" | ||
| } | ||
| } | ||
| }, | ||
| "discord": { | ||
| "type": "object", | ||
| "description": "description", | ||
| "properties": { | ||
| "webhook": { | ||
| "type": "string", | ||
| "description": "description", | ||
| "format": "uri-reference" | ||
| } | ||
| } | ||
| }, | ||
| "emails": { | ||
| "type": "array", | ||
| "description": "description", | ||
| "items": { | ||
| "type": "string", | ||
| "format": "email" | ||
| } | ||
| }, | ||
| "attach_csv": { | ||
| "type": "boolean", | ||
| "description": "description" | ||
| }, | ||
| "subject": { | ||
| "type": "string", | ||
| "description": "description" | ||
| }, | ||
| "skip_null": { | ||
| "type": "boolean", | ||
| "description": "description" | ||
| } | ||
| }, | ||
| "additionalProperties": false | ||
| } | ||
| }, | ||
| "required": ["id", "description", "search", "report"], | ||
| "additionalProperties": false | ||
| } | ||
| }, | ||
| "required": ["dag"], | ||
| "additionalProperties": false | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.