Tutorial Application
This tutorial covers features not covered in the basics.
Unlike the basic section, all items are independent. Create a new project individually and then follow the tutorial.
1. Conditional Branch
In a workflow, subsequent actions may change depending on the results of one action.
WHEEL provides an if component to achieve such conditional branching.
Let’s use the if component to run different scripts depending on the success or failure of the script.
First, create a new project. Then add two task components and an if component to the workflow. Each of the following is called task0, task1, if0.
First, create task0, task1 to run after branching from if0.
Create a new file for each component and enter the following in a text editor: In addition, specify the created file in the script property.
task0:
echo task0
task1:
echo task1
Output task0 or task1 to the standard output, respectively.
Then specify the conditional branch for the if component.
Open the properties screen, create a new file named check.sh, and open it in a text editor.
If the return value of check.sh is 0, it branches to a green ▼ (if clause), and if it is not 0, it branches to a purple ▼ (else clause). First,
exit 0
and make sure to pass the green ▼ (if clause).
Return to the component properties screen, click the condition setting line, and specify check.sh from the script name for condition check drop-down list.
Finally, connect the purple ▼ (else clause) displayed under if0 to task0 and the green ▼ (if clause) to task1.
The green connector (if clause) indicates components that advance when check.sh returns 0 and the purple connector (else clause) indicates components that advance when the return value is non-zero.
When you run this workflow, check.sh returns 0, so only task1 is run and stdout shows only task1.
Let’s initialize the project, change the contents of check.sh to exit 1, and try again.
This time, only task0 is executed, and stdout displays only task0.
2. Loop
The same process may be repeated in a workflow.
WHEEL has three loop components for (1), while (2), and foreach (3).
Within each component, you can define a nested workflow that iterates through subordinate workflows until the conditions specified for those components are met.
You can specify conditions for each component in the following ways:
- for
- Count up or down from the opening price to the closing price
- while
- Executes the specified shell script or javascript statement with a return value of 0 (true for javascript)
- foreach
- List all the values to be used
Let’s use the for component as a representative example to create a workflow that prints from 1 to 10.
First, add a for component to the project.
There is a text box in the for component property that specifies three numbers: start, end, and step. Specify 1, 5, and 2, respectively.
Then double-click the for component. The components in the for component are now visible, but there are no components now, so they are not visible.
Add one task component here. Also, add an empty file run.sh to the task component and specify it in the script property.
The task component in the for component has a $WHEEL_CURRENT_INDEX` environment variable that indicates the value of the loop variable. Open the file specified in the script in a text editor and write the following lines:
echo $WHEEL_CURRENT_INDEX
When you run the project, the standard output shows 1,3,5.
As shown in this example, internal components are executed when the value of the end property of the for component and the index variable ($WHEEL_CURRENT_INDEX) are equal.
3. Parameter Study
In a computer simulation workflow, you may want to perform the same calculation over and over again, varying some of the input data parameters incrementally.
WHEEL has a parameterStudy component (The following PS components) that supports such processing.
You can define nested workflows within PS components. By rewriting and executing any file in a subordinate workflow with a specified parameter value, you can perform the same operation multiple times with different parameters.
Let’s pass various arguments to the echo command by rewriting and executing the parameter values in the shell script.
First, add one PS component to the project.
Then double-click the PS component to see inside the PS component and add one task component.
Also, add an empty file called run.sh in the task component and specify it in the script property.
In run.sh
echo number
and write.
Save run.sh and return to the graph view. Click the project name (In the following example, tutorial_PS) at the top of the screen to return to the root component of the project.
Open the PS component properties screen to display the Files area. Click parameterSetting.json to go to the text editor screen with your selection.
The left side of the screen is the same text editor as before, but the right side of the screen provides a screen for setting up parameter studies.
First, click the + add new target file button at the top right of the screen. The component tree screen appears, click task0. Type run.sh for the file name and click the OK button.
The run.sh you just created appears in the text editor section. Drag the number part to select it.
The parameters text box on the right side of the screen displays number.
In this state, you can click the + add new parameter button to set the parameter for number. For now, change the drop-down list to list.
Next, set three values for foo bar baz. Type foo in the text box and click the + button. Follow the same steps to set bar and baz.
Finally, click the OK button.
The parameter settings for number are registered. The number part of run.sh is rewritten as {{ number }}.
At runtime, this {{ number }} is replaced by the actual parameters (foo, bar, baz) and executed respectively.
Finally, click the save all files button at the top right of the screen to save your edits.
When you run the project, stdout displays foo, bar, and baz, respectively, as output from the echo command.
About the order in which parameter studies are run
The PS component executes the subcomponents as parallel as possible for all combinations in the parameter space specified by parameterFile.
Therefore, the execution order of the lower components is out of order, and the output order of the results is out of order.
Addendum to Parameter Replacement
If {{ number }} appears elsewhere in the file specified in the targetfile, it will be replaced by a parameter as well.
You can also add filters within double parentheses. For example, {{ number | upper }} allows the parameter to be capitalized.
Another alternative is {{ “true” if number == “foo” else “false” }}, which is replaced by “true” only when number is foo, and “false” otherwise.
See here for available filters.
https://mozilla.github.io/nunjucks/templating.html#builtin-filters
4. Input File
Depending on the nature of the simulation, you may want preprocessing to be done interactively before the workflow is run, but you may want the workflow to automatically perform subsequent operations after the solver.
WHEEL provides a source component that uploads files at runtime and treats them as input data for the workflow.
Let’s learn about the source component by creating a workflow that uses the cat command to display any text file created on the client PC.
First, add a task component and a source component to your project.
Open the properties of the source component and click the upload setting line to enable the upload on demand switch. The output files of the source component displays UPLOAD_ON_DEMAND.
Then add an empty file called run.sh in the task component and specify it in the script property.
In run.sh, include the following:
cat input.txt
Finally, add input.txt to the input files of the task component and connect to UPLOAD_ON_DEMAND of the source component. This completes the workflow creation process.
In fact, prepare a text file on your client PC to use as an input file.
When you run the project, you will be prompted to upload a file to use as the outputfile for the source component. Specify the file you prepared earlier. When the upload is complete, the part that says drop file here changes to the name of the uploaded file.
Clicking the ok button resumes project execution and prints the contents of the file uploaded by the cat command to stdout.
You can upload the file again and again until you click the ok button. If you specify the wrong file, upload it again and the specified file will be used later.
5. Result Display
Workflows that are automated to post-process often output image files as the result of calculations.
WHEEL provides a viewer component that displays image files in typical formats in the browser. This component makes it easy to see the results without downloading the resulting file to the client PC.
Let’s learn about the viewer component by creating a workflow that passes a pre-prepared image file to the viewer component.
To get started, add a task component and a viewer component to your project.
Then add an empty file called run.sh in the task component and specify it in the script property.
In run.sh, include the following:
ls *
Uploads image data for display in the viewer component to the task component. Upload by dragging and dropping the image data to the Files area with the upload file button.
Specify the file name of the uploaded image data in the output files property.
By writing *.extension, you can specify all files with the same extension or omit the file name. (For example, for a JPEG file, write *.jpg)
If you want to display only one file as image data, you can specify the file name as it is.
It is also acceptable to specify multiple file names in the output files property without combining them into one.
Finally, add ./ to the input files of the viewer component and connect to the output files of the task component. This completes the workflow creation process.
When “./” is set for input files
Setting input files to ./ places all files passed as input files (In this case, *.JPG and *.web) directly under the viewer component directory.
When you run the project, you will see a dialog called open viewer screen.
Click the ok button to display the Image Viewer in a separate tab.
By default, the displayed image is tiled, but when you click the displayed image, it enters the slideshow state. In this state, you can zoom in and out of the image.
6. File Save
When you run a workflow with WHEEL, the resulting output files are deleted every time you initialize the project.
If you run the same workflow over and over with a human intervention, checking results and adjusting settings, you may want to save the results from each run.
In these cases, you can use the storage component to leave it undeleted during project initialization.
Let’s use the storage component to ensure that all files are saved by running and initializing the project multiple times while saving the files.
To get started, add one task component and one storage component to your project.
Then add an empty file called run.sh in the task component and specify it in the script property.
In run.sh, include the following:
echo foo >foo.txt
Also, add *.txt to the output files of the task component. This component only prints foo.txt, but the file name is specified as * because the project will now run several times while the output file name is rewritten.
Next, open the storage component properties window and write /tmp in the directory path field. Also, specify ./ for input files.
About setting directory path
If/tmp does not exist or/tmp does not have write permission, specify any other directory.
However, you must specify a path that is outside the scope of the directory tree where the project files are stored.
Finally, connect the output files of the task component and the intpuFile of the storage component. This completes the workflow creation process.
When you run the project, foo.txt appears in the Files area for each task0, storage0 component.
When you initialize the project, foo.txt for task0 is deleted, but foo.txt for storage0 remains.
If you edit run.sh in task0, rename the output file to bar.txt, and run it again, you will see foo.txt, bar.txt in the storage0 component.
In this way, when you initialize a project, the files stored in the storage component are retained, and can be used to store files for restart calculations.
7. Bulk Job
The bulk job function can be used in systems such as Fugaku that use Fujitsu’s HPC middleware “UJITSU Software Technical Computing Suite”.
WHEEL provides a bulkjobTask component to facilitate this functionality from the GUI.
Let’s use bulk jobs to create a workflow that is the same as the parameter study.
Before you begin, create a new project and add a builkjobTask component.
The default is to run on localhost, so change the property host to specify a remote host where bulkjob functionality is available.
Remote host settings when using the bulk job function
Note that even if the batch system running on the actual host supports the bulkjob function, the bulkjob function cannot be used unless use bulkjob is enabled in the remotehost setting.
Open the properties screen, create an empty file named run.sh, open a text editor, and write echo ${PJM_BULKNUM}.
When the job is executed, the ${PJM_BULKNUM} part is replaced with the set parameters.
Return to the properties screen and set script to run.sh.
Finally, click in the bulkjob setting row to specify the parameters. In this case, uncheck the use parameter setting file for bulk number and specify 0 for start and 5 for end.
Finally, to collect the standard output for each subjob, click the remote file setting line and specify run.sh* for include.
When the project has finished running, standard output is output with a file name of the form run.sh.xxxxx[${PJM_BULKNUM}].out .
Make sure that each PJM_BULKNUM matches the output of the echo command output to the file.
8. Step Job
The Step Job function can be used on systems using Fujitsu’s HPC middleware “FUJITSU Software Technical Computing Suite” such as Fugaku.
WHEEL provides the stepJob and stepJobTask components to facilitate this functionality from the GUI.
The stepjob function of WHEEL is to define multiple stepJobTask components within the stepJob component and define dependencies, and then execute them using the stepjob function of the batch system.
First, create a new project and add a stepJob component.
The default is to run on localhost, so change the property host to point to a remote host where stepjob functionality is available.
Remote host settings when using the step job function
Even if the batch system running on the actual host supports the step job function, the stepjob function cannot be used unless use stepjob is enabled in the remotehost setting.
Then double-click stepjob0 to view the components below stepjob0.
You can only place a stepjobTask component within a stepjob component. This switches the component library portion on the left side of the screen to show only the stepjobTask component.
Add two stepjobTask components here.
Because you have two components, sjTask0 and sjTask1, create an empty file named run.sh for each component and specify it in the script property.
The script for sjTask0 should be:
exit 1
This will cause sjTask0 to fail each time.
The script for sjTask1 must be:
echo stepjobTask1 > stdout.txt
It then specifies the order in which sjTask0 and sjTask1 run. Drag the ▼ of sjTask0 and drop it on sjTask1
The lines connect, and the icon on the left shoulder of sjTask1 changes to 1.
In addition, specify a run condition for sjTask1.
Click to open the stepjobtask setting area of the Properties window, enable use dependency, and enter sd=ec==1 for dependencyForm.
Now, sjTask1 is executed only when sjTask0 returns 1.
For more information about writing the dependencyForm, see the TCS documentation or the user guide for your computer system.
Finally, to retrieve the file output by sjTask1, click to open the remote file setting area and type stdout.txt in the include text box.
When I run the project, sjTask0 fails, but sjTask1 runs and prints stepjobTask1 in stdout.txt.
Once you have initialized the project, change the dependencyForm of sjTask1 to sd=ec==0 and run it.
This time, after sjTask0 fails, sjTask1 does not run and the entire workflow exits.
This concludes the application tutorial.
If you want to learn about the workflow that actually makes an application work, go to Practical Tutorial.
For detailed information about each function, refer to the Reference Manual.