Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graphql: Submit Workflow Execution #838

Merged
merged 37 commits into from
Jan 16, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
2d0a3ab
more fields
JeffreyThiessen Jan 5, 2024
f5678d8
cleanup, comments, fixes
JeffreyThiessen Jan 12, 2024
a5fa092
add test cases
JeffreyThiessen Jan 12, 2024
8a0ef82
add test for workflow executions samples resolver
JeffreyThiessen Jan 12, 2024
a3d4994
update tests
JeffreyThiessen May 10, 2024
69a1984
add new tests for new resolver
JeffreyThiessen May 10, 2024
a6a9936
add cleaned to type
JeffreyThiessen May 28, 2024
b0fb52b
add workflow namespace type connection
JeffreyThiessen Oct 29, 2024
b84bc01
attempt fixing namespace access
JeffreyThiessen Nov 1, 2024
55f498a
fix namespace fields
JeffreyThiessen Nov 1, 2024
765c4c0
add tests for workflow executions on groups and on projects
JeffreyThiessen Nov 1, 2024
a79459b
get samples through SamplesWorkflowExecutions instead of directly
JeffreyThiessen Nov 26, 2024
0eeb710
dump schema
JeffreyThiessen Dec 19, 2024
96ed44f
files added, not working yet
JeffreyThiessen Oct 29, 2024
91445fd
working logic
JeffreyThiessen Nov 6, 2024
577c058
Restructure code flow. no error handling yet.
JeffreyThiessen Nov 7, 2024
66253ef
error handling and attachment validation for workflow execution
JeffreyThiessen Nov 8, 2024
a0897b8
todo
JeffreyThiessen Nov 22, 2024
f46f490
restructure samples workflow execution attributes
JeffreyThiessen Dec 10, 2024
bfef0b6
add test cases
JeffreyThiessen Dec 12, 2024
2d10ce4
fix schema
JeffreyThiessen Dec 19, 2024
8edb02a
fix rebase errors
JeffreyThiessen Dec 19, 2024
869f7c2
fix rebase error duplicating tests
JeffreyThiessen Dec 19, 2024
b836961
fix one"
JeffreyThiessen Jan 7, 2025
6218817
fix tests hopefully
JeffreyThiessen Jan 7, 2025
471e58b
linting
JeffreyThiessen Jan 8, 2025
8ce4ba0
fix seeds not passing validation
JeffreyThiessen Jan 10, 2025
51490a0
fix new test case
JeffreyThiessen Jan 10, 2025
d6c9190
requested changes
JeffreyThiessen Jan 13, 2025
ec709b3
verify samples match internally and that attachments match samples
JeffreyThiessen Jan 15, 2025
ac5d6c1
add/update tests
JeffreyThiessen Jan 15, 2025
c0e3494
move validation from workflow_execution to samples_workflow_execution
JeffreyThiessen Jan 15, 2025
1f9d267
add missed file
JeffreyThiessen Jan 15, 2025
c2c9b61
update SamplesWorkflowExecutions tests to better reflect functionality
JeffreyThiessen Jan 15, 2025
3cc225b
rework some test fixtures affected by workflowexecution changes
JeffreyThiessen Jan 15, 2025
91abb5d
improve error messages
JeffreyThiessen Jan 15, 2025
d63ebe0
lint
JeffreyThiessen Jan 15, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion app/controllers/workflow_executions_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ def workflow_execution_params_attributes

def samples_workflow_execution_params_attributes
[
:id,
:id, # index, increment for each one, not necissary for functionality
:sample_id,
{ samplesheet_params: {} }
]
Expand Down
134 changes: 134 additions & 0 deletions app/graphql/mutations/submit_workflow_execution.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# frozen_string_literal: true

module Mutations
# Base Mutation
class SubmitWorkflowExecution < BaseMutation # rubocop:disable Metrics/ClassLength
null true
description 'Create a new workflow execution..'

argument :email_notification,
Boolean,
required: false,
default_value: false,
description: 'Set to true to enable email notifications from this workflow execution'
argument :name, String, required: false, description: 'Name for the new workflow.'
argument :samples_workflow_executions_attributes, [GraphQL::Types::JSON], description: "A list of hashes containing a 'sample_id', and a hash of `samplesheet_params`." # rubocop:disable GraphQL/ExtractInputType,Layout/LineLength
argument :update_samples, # rubocop:disable GraphQL/ExtractInputType
Boolean,
required: false,
default_value: false,
description: 'Set true for samples to be updated from this workflow execution'
argument :workflow_engine, String, description: '' # rubocop:disable GraphQL/ExtractInputType
argument :workflow_engine_parameters, [GraphQL::Types::JSON], description: 'List of Hashes containing `key` and `value` to be passed to the workflow engine.' # rubocop:disable GraphQL/ExtractInputType,Layout/LineLength
argument :workflow_engine_version, String, description: 'Workflow Engine Version' # rubocop:disable GraphQL/ExtractInputType
argument :workflow_name, String, description: 'Name of the pipeline to be run on this workflow execution' # rubocop:disable GraphQL/ExtractInputType
argument :workflow_params, GraphQL::Types::JSON, description: 'Parameters to be passed to the pipeline.' # rubocop:disable GraphQL/ExtractInputType
argument :workflow_type, String, description: 'Type of pipelines workflow.' # rubocop:disable GraphQL/ExtractInputType
argument :workflow_type_version, String, description: 'Version of the pipelines workflow type.' # rubocop:disable GraphQL/ExtractInputType
argument :workflow_url, String, description: 'Url for the pipeline.' # rubocop:disable GraphQL/ExtractInputType
argument :workflow_version, String, description: 'Version of the pipeline to be run on this workflow execution' # rubocop:disable GraphQL/ExtractInputType

# one of project/group, to use as the namespace
argument :project_id, ID, # rubocop:disable GraphQL/ExtractInputType
required: false,
description: 'The Node ID of the project to run workflow in. For example, `gid://irida/Project/a84cd757-dedb-4c64-8b01-097020163077`.' # rubocop:disable Layout/LineLength
argument :group_id, ID, # rubocop:disable GraphQL/OrderedArguments,GraphQL/ExtractInputType
required: false,
description: 'The Node ID of the group to run workflow in. For example, `gid://irida/Group/a84cd757-dedb-4c64-8b01-097020163077`.' # rubocop:disable Layout/LineLength
validates required: { one_of: %i[project_id group_id] }

field :errors, [Types::UserErrorType], null: false, description: 'A list of errors that prevented the mutation.'
field :workflow_execution, Types::WorkflowExecutionType, description: 'The newly created workflow execution.'

def resolve(args)
create_workflow_execution(args)
end

def ready?(**_args)
authorize!(to: :mutate?, with: GraphqlPolicy, context: { user: context[:current_user], token: context[:token] })
end

private

def create_workflow_execution(args) # rubocop:disable Metrics/MethodLength,Metrics/AbcSize
workflow_engine_parameters = build_workflow_engine_parameters(args[:workflow_engine_parameters])

update_samples = args[:update_samples] ? '1' : '0'
email_notification = args[:email_notification] ? '1' : '0'

namespace_id = namespace(args[:project_id], args[:group_id])

samples_workflow_executions_attributes = build_samples_workflow_executions_attributes(args[:samples_workflow_executions_attributes]) # rubocop:disable Layout/LineLength

workflow_execution_params = {
name: args[:name],
metadata: { workflow_name: args[:workflow_name],
workflow_version: args[:workflow_version] },
namespace_id:,
workflow_params: args[:workflow_params],
workflow_type: args[:workflow_type],
workflow_type_version: args[:workflow_type_version],
workflow_engine: args[:workflow_engine],
workflow_engine_version: args[:workflow_engine_version],
workflow_engine_parameters:,
workflow_url: args[:workflow_url],
update_samples:,
email_notification:,
samples_workflow_executions_attributes:
}

workflow_execution = WorkflowExecutions::CreateService.new(
current_user, workflow_execution_params
).execute

if workflow_execution.persisted?
{
workflow_execution:,
errors: []
}
else
user_errors = workflow_execution.errors.map do |error|
{
path: ['workflow_execution', error.attribute.to_s.camelize(:lower)],
message: error.message
}
end
{
workflow_execution: nil,
errors: user_errors
}
end
end

def namespace(project_id, group_id)
if project_id
IridaSchema.object_from_id(project_id, { expected_type: Project }).namespace.id
else # group_id
IridaSchema.object_from_id(group_id, { expected_type: Group }).id
end
end

# workflow engine parameters can have keys that start with `-` which is not allowed in graphql,
# so we parse a list of key value pairs into a hash that can be used.
def build_workflow_engine_parameters(parameter_list)
result = {}
parameter_list.each do |params|
result[params['key']] = params['value']
end
result
end

def build_samples_workflow_executions_attributes(samples_workflow_executions_attributes)
result = {}
samples_workflow_executions_attributes.each_with_index do |data, index|
sample_id = IridaSchema.object_from_id(data['sample_id'], { expected_type: Sample }).id
result[index.to_s] = {
'sample_id' => sample_id,
'samplesheet_params' => data['samplesheet_params']
}
end

result
end
end
end
115 changes: 115 additions & 0 deletions app/graphql/schema.graphql
Original file line number Diff line number Diff line change
Expand Up @@ -1416,6 +1416,16 @@ type Mutation {
input: CreateSampleInput!
): CreateSamplePayload

"""
Create a new workflow execution..
"""
submitWorkflowExecution(
"""
Parameters for SubmitWorkflowExecution
"""
input: SubmitWorkflowExecutionInput!
): SubmitWorkflowExecutionPayload

"""
Transfer a list of sample to another project.
"""
Expand Down Expand Up @@ -2857,6 +2867,111 @@ type SamplesWorkflowExecution implements Node {
workflowExecution: WorkflowExecution
}

"""
Autogenerated input type of SubmitWorkflowExecution
"""
input SubmitWorkflowExecutionInput {
"""
A unique identifier for the client performing the mutation.
"""
clientMutationId: String

"""
Set to true to enable email notifications from this workflow execution
"""
emailNotification: Boolean = false

"""
The Node ID of the group to run workflow in. For example, `gid://irida/Group/a84cd757-dedb-4c64-8b01-097020163077`.
"""
groupId: ID

"""
Name for the new workflow.
"""
name: String

"""
The Node ID of the project to run workflow in. For example, `gid://irida/Project/a84cd757-dedb-4c64-8b01-097020163077`.
"""
projectId: ID

"""
A list of hashes containing a 'sample_id', and a hash of `samplesheet_params`.
"""
samplesWorkflowExecutionsAttributes: [JSON!]!

"""
Set true for samples to be updated from this workflow execution
"""
updateSamples: Boolean = false

"""

"""
workflowEngine: String!

"""
List of Hashes containing `key` and `value` to be passed to the workflow engine.
"""
workflowEngineParameters: [JSON!]!

"""
Workflow Engine Version
"""
workflowEngineVersion: String!

"""
Name of the pipeline to be run on this workflow execution
"""
workflowName: String!

"""
Parameters to be passed to the pipeline.
"""
workflowParams: JSON!

"""
Type of pipelines workflow.
"""
workflowType: String!

"""
Version of the pipelines workflow type.
"""
workflowTypeVersion: String!

"""
Url for the pipeline.
"""
workflowUrl: String!

"""
Version of the pipeline to be run on this workflow execution
"""
workflowVersion: String!
}

"""
Autogenerated return type of SubmitWorkflowExecution.
"""
type SubmitWorkflowExecutionPayload {
"""
A unique identifier for the client performing the mutation.
"""
clientMutationId: String

"""
A list of errors that prevented the mutation.
"""
errors: [UserError!]!

"""
The newly created workflow execution.
"""
workflowExecution: WorkflowExecution
}

"""
Autogenerated input type of TransferSamples
"""
Expand Down
1 change: 1 addition & 0 deletions app/graphql/types/mutation_type.rb
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ class MutationType < Types::BaseObject
field :create_group, mutation: Mutations::CreateGroup # rubocop:disable GraphQL/FieldDescription
field :create_project, mutation: Mutations::CreateProject # rubocop:disable GraphQL/FieldDescription
field :create_sample, mutation: Mutations::CreateSample # rubocop:disable GraphQL/FieldDescription,GraphQL/ExtractType
field :submit_workflow_execution, mutation: Mutations::SubmitWorkflowExecution # rubocop:disable GraphQL/FieldDescription
field :transfer_samples, mutation: Mutations::TransferSamples # rubocop:disable GraphQL/FieldDescription
field :update_sample_metadata, mutation: Mutations::UpdateSampleMetadata # rubocop:disable GraphQL/FieldDescription
end
Expand Down
2 changes: 2 additions & 0 deletions app/models/samples_workflow_execution.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,6 @@ class SamplesWorkflowExecution < ApplicationRecord
belongs_to :sample
has_many_attached :inputs
has_many :outputs, dependent: :destroy, class_name: 'Attachment', as: :attachable

validates_with WorkflowExecutionSamplesheetParamsValidator
end
2 changes: 1 addition & 1 deletion app/models/workflow_execution.rb
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ def send_email
end

def cancellable?
%w[submitted running prepared initial].include?(state)
%w[submitted running prepared initial].include?(state)
end

def deletable?
Expand Down
39 changes: 39 additions & 0 deletions app/validators/workflow_execution_samplesheet_params_validator.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# frozen_string_literal: true

# Validator for Workflow Execution Samplesheet Params
# This will cause the validation to fail if any of the attachment ids cannot be resolved
class WorkflowExecutionSamplesheetParamsValidator < ActiveModel::Validator
def validate(record) # rubocop:disable Metrics/MethodLength,Metrics/AbcSize,Metrics/CyclomaticComplexity,Metrics/PerceivedComplexity
record.samplesheet_params.each do |key, value| # rubocop:disable Metrics/BlockLength
if key == 'sample'
if value.nil? || value == ''
error_message = 'No Sample PUID provided'
record.errors.add :sample, error_message
record.workflow_execution.errors.add :sample, error_message
elsif value != (record.sample.puid)
error_message = "Provided Sample PUID #{value} does not match SampleWorkflowExecution Sample PUID #{record.sample.puid}" # rubocop:disable Layout/LineLength
record.errors.add :sample, error_message
record.workflow_execution.errors.add :sample, error_message
end
next
end

next if value == ''

begin
# Attempt to parse an object from the id provided
attachment = IridaSchema.object_from_id(value, { expected_type: Attachment })
unless attachment.attachable == record.sample
error_message = "Attachment does not belong to Sample #{record.sample.puid}."
record.errors.add :attachment, error_message
record.workflow_execution.errors.add :attachment, error_message
end
rescue StandardError => e
error_message = e.message
record.errors.add :attachment, error_message
record.workflow_execution.errors.add :attachment, error_message
next
end
end
end
end
21 changes: 18 additions & 3 deletions db/seeds.rb
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,15 @@ def seed_workflow_executions # rubocop:disable Metrics/MethodLength, Metrics/Abc
workflow_engine_version: '23.10.0',
workflow_engine_parameters: { '-r': 'dev' },
workflow_url: 'https://github.com/phac-nml/iridanextexample',
submitter: User.find_by(email: '[email protected]')
submitter: User.find_by(email: '[email protected]'),
samples_workflow_executions_attributes: {
'0': {
sample_id: Sample.first.id,
samplesheet_params: {
sample: Sample.first.puid
}
}
}
)

SamplesWorkflowExecution.create(
Expand All @@ -188,7 +196,15 @@ def seed_workflow_executions # rubocop:disable Metrics/MethodLength, Metrics/Abc
workflow_url: 'https://github.com/phac-nml/iridanextexample',
submitter: User.find_by(email: '[email protected]'),
blob_run_directory: 'this should be a generated key',
state: :completed
state: :completed,
samples_workflow_executions_attributes: {
'0': {
sample_id: Sample.first.id,
samplesheet_params: {
sample: Sample.first.puid
}
}
}
)

filename = 'summary.txt'
Expand All @@ -197,7 +213,6 @@ def seed_workflow_executions # rubocop:disable Metrics/MethodLength, Metrics/Abc
attachment.save!

SamplesWorkflowExecution.create(
samplesheet_params: { my_key1: 'my_value_2', my_key2: 'my_value_3' },
sample: Sample.first,
workflow_execution: workflow_execution_completed
)
Expand Down
2 changes: 1 addition & 1 deletion test/controllers/workflow_executions_controller_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ class WorkflowExecutionsControllerTest < ActionDispatch::IntegrationTest
{
sample_id: @sample1.id,
samplesheet_params: {
sample: "Sample_#{@sample1.id}",
sample: @sample1.puid,
'fastq_1' => @attachment1.to_global_id,
'fastq_2' => ''
}
Expand Down
Loading
Loading