Skip to content

README.md

Latest commit

 

History

History
204 lines (136 loc) · 14 KB

README.md

File metadata and controls

204 lines (136 loc) · 14 KB

ExaGO visualization (experimental)

ExaGO has an experimental visualization platform for visualizing the results of OPFLOW on a map provided the geospatial coordinates for the network are available. On launching the visualization, a webpage displays the given power system network overlayed on a geospatial map. The (experimental) visualization features include:

  • Map-based network layout of the grid
  • Reading in the grid data through a geojson file.
  • Fly-in (zoom in) on bus, branch, county
  • County-based load and voltage heatmaps
  • Bar chart layer and double pie-chart for generation dispatch and capacity
  • Filters based on network voltage, generation dispatch level, and voltage/load level
  • Zoomed-in display of county load or voltage (aggregated)

Enhanced features

The following features have been added on top of the base visualization:

Settings panel

A persistent settings dialog (gear icon in the control panel) allows users to configure visualization options without reloading. Settings are saved to the browser's local storage and restored on next launch. Configurable options include:

  • Map style — switch between Positron, Positron (no labels), Dark Matter, or no basemap
  • Active power flow layer — toggle animation, clustering, and adaptive scales; set gradient colour (low → high loading) and overall layer opacity
  • Reactive power flow layer — same options as the active flow layer, with independent colour and opacity controls
  • Network display — adjust node circle radius (px), line width scale, and toggle/colour county and state boundary overlays
  • Generation bars — set bar radius (km), height scale (MW multiplier), and enable zoom-adaptive radius so bars maintain a consistent on-screen size as the user zooms in and out

Hover tooltips

Rich hover tooltips appear when the cursor moves over network elements:

  • Bus (substation) — shows voltage level (kV), voltage magnitude (p.u.), voltage angle (°), real/reactive load (MW/MVAR), bus type, area, and zone
  • Transmission line — shows voltage (kV), active/reactive power from/to (MW/MVAR), thermal rating (MW), and P loading % with colour coding (green < 70%, amber 70–90%, red > 90%)
  • Generator bar — shows generator name, fuel type, bus, area, zone, output (Pg MW), capacity (Pcap MW), loading %, reactive power (Qg MVAR), and voltage setpoint (Vg p.u.); all fields render conditionally based on data availability

Control panel

All controls are consolidated into a single right-hand panel (300 px wide):

  • Case selector — dropdown to switch between pre-loaded network cases; the map automatically flies to the extent of the newly loaded dataset while preserving the current camera pitch
  • File upload — upload any compatible .json case file directly from the panel
  • Screenshot — capture the current map view (including all active layers) as a PNG, preview the thumbnail in the panel, and download with a single click

Installation

ExaGO visualization uses the following tools to generate the visuals.

Before launching the visualization, one needs to install these packages. This can be done with the following steps:

  1. Install Node Version Manager (NVM). On MAC use brew install nvm.
  2. Install [Node.js] version 24 (https://nodejs.org/en/) using nvm install 24
  3. Select Node 24 by nvm use 24
  4. Install Yarn. Do npm install --global yarn
  5. Do yarn install in this directory (viz) to install all the dependencies.
  6. Go to the viz/backend subdirectory and use the pip install -r requirements.txt command to install all the Python dependencies.

Preparing input data files for visualization

The visualization uses a JSON formatted file as an input. This JSON file has a specific structure (To do: explain structure for the file) and there are several sample files for different network in the data subdirectory. This input JSON file can be either created externally OR generated as an output of the OPFLOW application. When using OPFLOW, the following command will generate the input JSON file. The generated file will be name as opflowout.json.

./opflow -netfile <netfile> -save_output -opflow_output_format JSON -gicfile <gicfilename>

Note that the OPFLOW application is available in the $EXAGO_INSTALL/bin directory where $EXAGO_INSTALL is the ExaGO installation directory.

The above command will run a OPFLOW on the given network and generate an output file called opflowout.json. The -gicfile is an additional option one can provide to provide the file that has the geospatial coordinates (latitude/longitude) for the network. If the geospatial coordinates are not provided then OPFLOW draws the network as a circle. It is highly recommended that one provides the geospatial coordinate file as an input to display the network correctly on the map. The geospatial coordinate file should have the same format as used for the Electric Grid Test Case Repository synthetic networks.

For example, with Texas 2000 bus synthetic data, executing the following opflow will produce the opflowout.json output. The case files are provided in the data folder.

opflow -netfile case_ACTIVSg2000.m -save_output -opflow_output_format JSON -gicfile ACTIVSg2000_GIC_data.gic

Next, you can put the opflowout.json file in the viz/data folder. When the visualization tool will be launched, it will find all *.json files in the viz/data folder and show a list of files in the top right corner. The first item in the list will be visualized as default. Users can change the selection and the visualization will be updated accordingly. In addition, users can upload a compatible json case file (generated via opflow) using the file upload button next to the selection list.

Now you are ready to launch the visualization now.

Launch visualization

To launch the visualization, run

yarn start

This will open a webpage (e.g. http://localhost:5173/) on the default browser with the visualization of the given network. If the network is large, it may take a while to load the visualization. The browser may show option to terminate or Wait and you should click on Wait button.

The figures show the visualization of the synthetic electric grid. The data for developing this visualization was created by merging the synthetic dataset for the Eastern, Western, and Texas interconnects from the Electric Grid Test Case Repository

2D synthetic US western grid network display

2D synthetic US western grid tranmission line flow display

2.5D synthetic US western grid network display with generation overlapped and doughnut chart for generataion mix

2.5D synthetic US western grid displaying load profile by counties

2.5D synthetic US western grid displaying network, flow, generation, and load

Demo

See Here

ChatGrid

ChatGrid is a natural language query tool for ExaGO visualizations. It is powered by OpenAI GPT-3.5-Turbo and Langchain. ChatGrid allows users to query on ExaGO visualizations through natural language and returns text summaries and visual outputs as answers. The following flow chart shows the architecture design of ChatGrid.

Dependencies

ChatGrid is built upon the following services and tools.

Preparing the database for query

Behind the scenes, LLM translates natural language queries into SQL queries to retrieve data from a database. As the power grid network is a typical geospatial dataset, we choose PostgreSQl + PostGIS database for the convenience of conducting spatial queries. Please follow the steps below to set up your PostgreSQL database that contains the power grid network dataset.

  1. Convert data formats.

    First, we need to convert the ExaGO output .json files to .csv files. The difference between the two data formats is that JSON stores attributes and values as dictionary pairs but CSV stores attributes and values as tables. You can write your own script for this conversion or use the provided script.

    • Go to the viz/backend subdirectory and use the pip install -r requirements.txt command to install all the Python dependencies if already not done in previous steps. (Note: These steps are tested with Python 3.13.)

    To use the provided script, ensure that the ExaGO output .json file is in the viz/data subdirectory (if not already performed in the previous steps) and run the following script in the viz/backend subdirectory (replace the example filename with your json filename). This will create three CSV files: generation.csv, bus.csv, and tranmission_line.csv. We are assuming opflowout.json is the data json file present in viz/data folder.

    cd viz/backend
python ../data/jsontocsv.py ../data/opflowout.json

    Now there should be 5 CSV files in viz/backend folder:

    • bus.csv
    • generation.csv
    • us_states.csv
    • counties.csv
    • transmission_line.csv

  2. Download PostgreSQL database from this link and install it.

    • For MAC using brew you can install postgresql 14 using: brew install postgresql@14
    • Start the postgressql service: brew services start postgresql@14
    • Create a role: psql -U "$USER" -d postgres
    • If your username ($USER) starts with numbers psql will show error. In this case replace $USER to some other username.
    • Execute the create role query: CREATE ROLE postgres WITH LOGIN SUPERUSER PASSWORD 'ExaGO.2025'; Here ExaGO.2025 is a password. Change to your preference.
      • If you forget the password, you can update it by running ALTER USER user_name WITH PASSWORD 'new_password'; (replace user_name and new_password)
    • Exit to shell by entering quit and hitting Enter.
    • From command prompt type: psql -U postgres -d postgres If it works and you are in psql shell you are done. Exit from the shell using quit.

  3. Create a PostgreSQL database and import the .csv files to it.

    There are many ways to do that. If you are not familiar with SQL commands, you can use pgAdmin 4 (a GUI management program of PostgreSQL database, usually installed along with PostgreSQL) and follow this instruction to import the .csv files (e.g., generation.csv, bus.csv, tranmission_line.csv, counties.csv, US states.csv) to PostgreSQL database.

    Note:

    a. Please write down your username, password, database name and keep them in a safe place. You will need this information to connect to the database later.

    b. Please be informative and accurate about your table names, and attribute names. Because this information can help LLM understand the dataset and performs better when dealing with user queries.

    c. Include US state and county information in your database to support spatial queries that related to state or county.

    d. To enter the CSV files into database using command prompt do: PGPASSWORD=ExaGO.2025 ./create_db.sh --db exago_db --schema-sql ./schema.sql --drop --truncate. Here exago_db is the database name. Use it in the configuration config.py file.

     * If you used different user name than `$USER` then use the following command: `PGUSER=user_name PGPASSWORD=ExaGO.2025 ./create_db.sh  --db exago_db --schema-sql ./schema.sql --drop  --truncate` (replace `user_name` with your user name.)

    e. This will create a database named exago_db with password ExaGO.2025. This information will be used to update the config.py file.

  4. Connect to your database.

    Open the config.py file in the viz/backend subdirectory and replace YOUR_DATABASE_PASSWORD and YOUR_DATABASE_NAME with your own database password and database name.

Getting your OpenAI API key

ChatGrid uses GPT models from OpenAI to process natural language queries. To use LLMs from OpenAI, you first need to go to OpenAI's Platform website and sign in with an OpenAI account. Click your profile icon at the top-right corner of the page and select "View API Keys." Click "Create New Secret Key" to generate a new API key.

Open the config.py in the viz/backend subdirectory replace YOUR_OPENAI_KEY with your own OpenAI API key.

Launch backend

ChatGrid uses Flask to host the service of receiving user queries and returning the data output and text summaries to update the visualizations on the frontend. Please follow the steps below to run the backend server.

  • Run the following command in the viz/backend subdirectory

    python server.py

    This will start the backend server for receiving user queries, processing it by LLM and returning data outputs to the frontend.

Now open the chat window on the frontend, type your queries, and enjoy ChatGrid!

Optional: model configuration

If you would like to test different LLMs with ChatGrid, you can specify the model_name="YOUR_LLM_MODEL" in the viz/backend/sqlchain.py file.