Skip to content

Restructured specs and task package auto-generation script #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 42 commits into from
Nov 1, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
73d1e96
implement restructure of specs to add additional level
tclose Jul 25, 2023
2cbdfad
added hep strings to spec metadata and redesigned generated test
tclose Jul 25, 2023
e62f636
add conftest to generated test directory
tclose Jul 25, 2023
292f61d
reworked extract ROI example spec
tclose Jul 25, 2023
83b2f22
add conftest into generated package directories
tclose Jul 25, 2023
adf8c4e
Restructured yaml stubs in created packages so that they include comm…
tclose Jul 26, 2023
07b3321
fine tuning generated conversion spec
tclose Jul 26, 2023
db5d299
renamed nipype_name to new_name
tclose Jul 26, 2023
aa12798
finished fine-tuning conversion specs
tclose Jul 26, 2023
79c1b9c
added doc string to generated yaml comments
tclose Jul 26, 2023
ce1111f
removed stray input/output interfaces from to-import list
tclose Jul 26, 2023
c52d74e
touching up create_packages
tclose Jul 26, 2023
23eae0c
updated to use fileformats-datascience package
tclose Jul 27, 2023
30751fd
updated generate_sample_data method name
tclose Jul 27, 2023
d2357ac
create_packages now reads every docstring
tclose Jul 27, 2023
67b5c97
debugging created packages
tclose Jul 27, 2023
bb3e369
changed parsing of union file types
tclose Jul 29, 2023
a299ec1
added timeout_pass decorator
tclose Jul 29, 2023
5deffbc
moved github workflows into separate directory
tclose Jul 29, 2023
832225e
added auto-convert requirements
tclose Jul 29, 2023
0418bd2
changed create_packages to read multiple doctests
tclose Jul 29, 2023
38b9980
handle imports in doctests and unittests
tclose Jul 31, 2023
be84dae
touching up package initialisation in create_packages
tclose Jul 31, 2023
cb5deb2
debugged input typing
tclose Jul 31, 2023
4202323
debugged auto creation of packages
tclose Aug 1, 2023
913e3a2
added new field for task name
tclose Aug 1, 2023
7044f65
removed 'new_name' field from stubs
tclose Aug 1, 2023
5d91c8e
various debugging
tclose Aug 1, 2023
2e15096
finished writing template README
tclose Aug 1, 2023
f4416a7
removed timeout_pass
tclose Aug 1, 2023
37b1903
touched up readme and conversions
tclose Aug 1, 2023
4faa0b8
updated example specs
tclose Aug 1, 2023
e81a4fe
touched up package-gen README and converted to RST
tclose Aug 1, 2023
45a105a
touched up readme and template readme
tclose Aug 2, 2023
1e8f65d
added descriptions to input and output field references
tclose Aug 3, 2023
f72eca2
added dependency on nipype-conv step in GH action workflow
tclose Aug 3, 2023
f6439ee
added option to select new yaml file with list of packages to import
tclose Aug 16, 2023
674910b
updated tests badge
tclose Aug 16, 2023
54f8c31
commented out failing conftest copy
tclose Aug 16, 2023
ca60a9e
added fileformats-datascience to deps
tclose Aug 16, 2023
4ecf344
updated import location for fileformats Xml and Dicom
tclose Sep 5, 2023
218acd3
fixed bug related to updated fileformats to_mime
tclose Nov 1, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 40 additions & 15 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,21 +47,46 @@ specification,

.. code-block:: yaml

task_name: Registration
nipype_module: nipype.interfaces.ants.registration
output_requirements:
output_warped_image: ["fixed_image", "moving_image", "output_transform_prefix"]
output_templates:
output_warped_image: "{output_transform_prefix}warped"
doctest:
fixed_image: test.nii.gz
moving_image: test.nii.gz
cmdline: >-
antsRegistration --output [ output_, output_warped_image.nii.gz ]
--metric Mattes[ test.nii, test.nii, 1, 32, Random, 0.05 ]
tests_inputs: []
tests_outputs:
- AttributeError
task_name: n4_bias_field_correction
nipype_name: N4BiasFieldCorrection
nipype_module: nipype.interfaces.ants.segmentation
inputs:
omit:
# list[str] - fields to omit from the Pydra interface
rename:
# dict[str, str] - fields to rename in the Pydra interface
types:
# dict[str, type] - override inferred types (use "mime-like" string for file-format types,
# e.g. 'medimage/nifti-gz'). For most fields the type will be correctly inferred
# from the nipype interface, but you may want to be more specific, particularly
# for file types, where specifying the format also specifies the file that will be
# passed to the field in the automatically generated unittests.
input_image: medimage/nifti1
mask_image: medimage/nifti1
weight_image: medimage/nifti1
bias_image: medimage/nifti1
metadata:
# dict[str, dict[str, any]] - additional metadata to set on any of the input fields (e.g. out_file: position: 1)
outputs:
omit:
# list[str] - fields to omit from the Pydra interface
rename:
# dict[str, str] - fields to rename in the Pydra interface
types:
# dict[str, type] - override inferred types (use "mime-like" string for file-format types,
# e.g. 'medimage/nifti-gz'). For most fields the type will be correctly inferred
# from the nipype interface, but you may want to be more specific, particularly
# for file types, where specifying the format also specifies the file that will be
# passed to the field in the automatically generated unittests.
output_image: medimage/nifti1
bias_image: medimage/nifti1
callables:
# dict[str, str] - names of methods/callable classes defined in the adjacent `*_callables.py`
# to set to the `callable` attribute of output fields
templates:
# dict[str, str] - `output_file_template` values to be provided to output fields
requirements:
# dict[str, list[str]] - input fields that are required to be provided for the output field to be present

*Detailed description of the different options to go here*

Expand Down
6 changes: 6 additions & 0 deletions conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import tempfile
import pytest
from click.testing import CliRunner
from fileformats.generic import File


PKG_DIR = Path(__file__).parent
Expand All @@ -12,6 +13,11 @@
EXAMPLE_WORKFLOWS_DIR = EXAMPLE_SPECS_DIR / "workflow"


@pytest.fixture
def gen_test_conftest():
return PKG_DIR / "scripts" / "pkg_gen" / "resources" / "conftest.py"


@pytest.fixture(params=[str(p.stem) for p in (EXAMPLE_TASKS_DIR).glob("*.yaml")])
def task_spec_file(request):
return (EXAMPLE_TASKS_DIR / request.param).with_suffix(".yaml")
Expand Down
Loading