README

FHIR JSON Flattening and Upload to S3

About FHIR Data

Fast Healthcare Interoperability Resources (FHIR®) is a draft standard from Health Level 7 International (HL7) designed to allow the exchange of electronic health records. It is built upon previous standards such as HL7 version 2 and HL7 version 3, and uses XML and JSON for data representation. FHIR® solutions can be created using components called Resources which are built around particular concepts, such as the patient or provider, as well as diagnostics and care plans. Systems implementing FHIR® must support Resources in the XML format, but may also support Resources in the JSON format.

Reference : https://www.ibm.com/docs/ja/integration-bus/10.0?topic=SSMKHH_10.0.0/com.ibm.healthcare.pattern.FHIRTransformation.doc/pattern/overview.html

About this repo

This repository contains a Python script for fetching FHIR JSON data from an API, flattening it, and uploading the flattened data to an Amazon S3 bucket.

Prerequisites

Before running the script, ensure you have the following dependencies installed:

• Python 3.x
• requests
• boto3

You can install the dependencies via pip:

pip install requests boto3

Configuration

Make sure to update the following information in the script:

• config.json: Provide your AWS credentials (AWS Access Key ID and AWS Secret Access Key).
• api_url: URL of the API endpoint from which to fetch FHIR JSON data.
• bucket_name: Name of the S3 bucket where the flattened data will be uploaded.
• polling_interval: Interval (in seconds) for polling the API and uploading data to S3.

Usage

1. Clone the repository:

git clone https://github.com/your-username/fhir-json-flattening.git
cd FHIR-JSON-Flattening-and-Upload-to-S3

2. Update the config.json file with your AWS credentials.

3. Run the Python script main.py:

python main.py

Description

This script periodically fetches FHIR JSON data from the specified API endpoint, flattens it, and uploads the flattened data to an Amazon S3 bucket. It follows a polling mechanism with a specified interval to ensure regular updates.

Input

• FHIR JSON data fetched from the API specified in api_url.

Output

• Flattened FHIR JSON data uploaded to the specified S3 bucket.
• Logs indicating the status of data retrieval, flattening, and uploading.

Contributing

Contributions are welcome! If you have suggestions, feature requests, or bug fixes, please feel free to open an issue or create a pull request.

Acknowledgements

• requests - HTTP library for Python.
• boto3 - AWS SDK for Python.

Workflow

Step 1:

I have created a local JSON server, which is available as a NPM package. By running “ npm install -g json-server ” command, we can install the server in our local machine. Then we need to create a JSON file. Here I have taken a dummy FHIR JSON data as ‘db.json’ . After that we need to start the server with “ json-server --watch db.json “ this command.

Db.json :

Starting the server :

Now we are ready with our Json server.

Step 2 :

Now it’s time to test the server. To test the server I have used POSTMAN software. Postman Interface :

Here we have many options. When I opt GET, I got the db.json’s data as output. There is another option like PUT by that we can put new Json in our server.

Reference : Create A REST API With JSON Server | by Sebastian | CodingTheSmartWay | Medium

Step 3 :

After creating and testing the server I build a python framework, which can read the Json data from our server. After successfully got the data it can upload the data in S3 bucket by creating a unique file name every time. After uploading the Json file, it will flatten the Json data and will upload the flatten data in another S3 bucket as txt file. After doing all the process it will wait for 10 mins after that it’ll repeat the process again.

Output:

Json file in S3 bucket :

Flatten Output in S3 Bucket:

Flattened object’s output :