Creating Workflows

Fire Insights enables users to define end-to-end workflows for data pipelining leveraging pre-packaged nodes for common ETL and Machine Learning models. Workflows are then saved and executed to produce results. Sparkflows provides a a very intuitive and user friendly editor to achieve the same.

Define New Workflow

Click on ‘Create New Workflow’ for creating a New Workflow, It supports two engines - spark & pyspark. It will open the Workflow Editor where the workflow can be created.

workflow

Adding New Nodes to the Workflow

  • Workflows editor has a list of Nodes menu on the LHS. Clicking on any of the Nodes creates it in the workspace.

Creating Edges

  • Nodes can be connected by edges.
  • Click on the orange box and drag to the next node to create an edge between them.

Deleting Edges

  • Edges can be deleted by double clicking on them.

Saving Workflow

  • Give the workflow a name.
  • Click on the Save button to create the new workflow.

View Workflows

You can view the workflows by going to the Workflows Page inside specific applications.

workflow