Skip to content

#18 Add ignore_missing option on FileFetchTask that allow throwing Ex… #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 31, 2025
Merged

#18 Add ignore_missing option on FileFetchTask that allow throwing Ex… #19

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions docs/reference/tasks/01-FileFetchTask.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,13 @@ Filename of copied file.
Options
-------

| Code | Type | Required | Default | Description |
|--------------------------|:----------:|:---------:|:---------:|----------------------------------------------------------------------------------------------------------------------------------------------|
| `source_filesystem` | `string` | **X** | | The source flysystem/storage.<br/>See config/packages/flysystem.yaml to see configured flysystem/storages. |
| `destination_filesystem` | `string` | **X** | | The source flysystem/storage.<br/>See config/packages/flysystem.yaml to see configured flysystem/storages. |
| `file_pattern` | `string` | | null | The file_pattern used in preg_match to match into `source_filesystem` list of files. If not set try to use input as strict filename to match |
| `remove_source` | `bool` | | false | If true delete source file after copy |
| Code | Type | Required | Default | Description |
|--------------------------|:--------:|:--------:|:-------:|----------------------------------------------------------------------------------------------------------------------------------------------|
| `source_filesystem` | `string` | **X** | | The source flysystem/storage.<br/>See config/packages/flysystem.yaml to see configured flysystem/storages. |
| `destination_filesystem` | `string` | **X** | | The source flysystem/storage.<br/>See config/packages/flysystem.yaml to see configured flysystem/storages. |
| `file_pattern` | `string` | | null | The file_pattern used in preg_match to match into `source_filesystem` list of files. If not set try to use input as strict filename to match |
| `remove_source` | `bool` | | false | If true delete source file after copy |
| `ignore_missing` | `bool` | | true | Ignore property accessor errors for this source |


Examples
Expand All @@ -43,6 +44,7 @@ Examples
- copy all .csv files from 'storage.source' to 'storage.destination'
- remove .csv from 'storage.source' after copy
- output will be filename of copied files
- throw Exception when file(s) not found
```yaml
# Task configuration level
code:
Expand All @@ -52,6 +54,7 @@ code:
destination_filesystem: 'storage.destination'
file_pattern: '/.csv$/'
remove_source: true
ignore_missing: false
```

* Simple fetch process configuration to cipy a specific file from --input option via <br> ```bin/console cleverage:process:execute my_custom_process --input=foobar.csv -vv```
Expand Down
6 changes: 6 additions & 0 deletions src/Task/FileFetchTask.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,9 @@ protected function findMatchingFiles(ProcessState $state): void
$this->matchingFiles[] = $input;
}
}
if ([] === $this->matchingFiles && !$this->getOption($state, 'ignore_missing')) {
throw new \UnexpectedValueException('File(s) not found in source filesystem');
}
}

/**
Expand Down Expand Up @@ -166,5 +169,8 @@ protected function configureOptions(OptionsResolver $resolver): void

$resolver->setDefault('remove_source', false);
$resolver->setAllowedTypes('remove_source', 'boolean');

$resolver->setDefault('ignore_missing', true);
$resolver->setAllowedTypes('ignore_missing', 'boolean');
}
}