|
| 1 | +--- |
| 2 | +title: "Composing AI Inference workflows based on OSCAR services with Elyra in EGI Notebooks" |
| 3 | +date: 2024-04-15T09:00:00+01:00 |
| 4 | +# post image |
| 5 | +image: "/images/blog/post-elyra-egi-notebooks/elyra_icon.png" |
| 6 | +# post type (regular/featured) |
| 7 | +type: "featured" |
| 8 | +# meta description |
| 9 | +description: "A step-by-step guide to using Elyra within EGI Notebooks to interact with OSCAR services." |
| 10 | +# post draft |
| 11 | +draft: false |
| 12 | +--- |
| 13 | + |
| 14 | +{{< youtube 1pFjs0LND4E >}} |
| 15 | + |
| 16 | + |
| 17 | +# What is Elyra and EGI Notebooks? |
| 18 | + |
| 19 | + |
| 20 | + |
| 21 | +[Elyra](https://elyra.readthedocs.io/en/latest/index.html) is an open-source extension for [Jupyter Notebooks](https://jupyter.org/), designed to enhance the workflow of AI projects by supporting machine learning pipelines visually. It integrates seamlessly with various systems for orchestrating workflows, allowing users to create, run, and monitor their workflows within Jupyter environments. [EGI Notebooks](https://notebooks.egi.eu/hub/welcome), provided by the EGI cloud services, offer a cloud-based environment for running Jupyter Notebooks. This service simplifies the deployment and scaling of computational notebooks, providing researchers with powerful computational resources and a collaborative space for their scientific inquiries. |
| 22 | + |
| 23 | + |
| 24 | +# How a workflow in Elyra works? |
| 25 | + |
| 26 | +In Elyra, a workflow is a set of nodes or steps, each representing a script or notebook, connected in a flow that defines the execution order. Users can drag and drop nodes to create a visual representation of their workflow. Each node can be configured with specific parameters, inputs, and outputs. When a pipeline is run, Elyra executes the nodes sequentially or in parallel, depending on their dependencies. This visual approach not only simplifies the understanding and editing of machine learning workflows but also integrates with platforms like Kubeflow Pipelines and Apache Airflow to manage the execution in scalable environments. |
| 27 | + |
| 28 | + |
| 29 | + |
| 30 | + |
| 31 | + |
| 32 | + |
| 33 | +# Deploying a pipeline |
| 34 | + |
| 35 | +Deploying a pipeline in Elyra involves several key steps: |
| 36 | + |
| 37 | +1. Designing the Pipeline: Start by organizing your scripts or notebooks into nodes within the Elyra pipeline editor. |
| 38 | +2. Configuring Nodes: Set up each node with the necessary parameters, such as file paths, environment variables, and dependencies. |
| 39 | +3. Connecting Nodes: Connect the nodes to define the flow of data and execution sequence. |
| 40 | +4. Running the Pipeline: Execute the pipeline directly from Elyra, which will orchestrate the workflow based on the configured platform, such as local, Kubeflow, or Airflow. |
| 41 | +5. Monitoring Execution: Elyra provides tools to monitor the pipeline’s progress and debug if needed, ensuring that any issues can be addressed promptly. |
| 42 | + |
| 43 | +This process enables users to efficiently manage complex workflows and automate repetitive tasks, enhancing productivity and focusing on strategic tasks. |
| 44 | + |
| 45 | + |
| 46 | +# Summary |
| 47 | + |
| 48 | +Elyra simplifies the integration of OSCAR services within the EGI Notebooks environment, offering a user-friendly approach to managing complex pipelines. This post outlines how Elyra enhances productivity by streamlining processes, from repository cloning to workflow execution. |
| 49 | + |
| 50 | +We hope this overview inspires you to explore the capabilities of Elyra and OSCAR in EGI Notebooks, helping you to leverage these powerful tools for your projects. Enjoy the innovative computing possibilities! |
| 51 | + |
| 52 | + |
| 53 | + |
| 54 | + |
| 55 | + |
| 56 | + |
0 commit comments