Skip to content

Commit 60918ff

Browse files
Merge pull request #341 from smilingprogrammer/week-6-docs
chore(docs): Data pipeline for safaa week 6 blog, and added images to… Reviewed-by: shaheem.azmal@siemens.com
2 parents 275319e + 34b3920 commit 60918ff

File tree

9 files changed

+44
-1
lines changed

9 files changed

+44
-1
lines changed

docs/2025/data-pipeline/updates/2025-06-11.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ SPDX-FileCopyrightText: 2025 Abdulsobur Oyewale <oyewaleabdulsobur@gmail.com>
1111
-->
1212

1313
# WEEK 2
14-
*(June 11, 2024)*
14+
*(June 11, 2025)*
1515

1616
## Attendees:
1717
- [Shaheem Azmal M MD](https://github.com/shaheemazmalmmd)
@@ -24,6 +24,9 @@ SPDX-FileCopyrightText: 2025 Abdulsobur Oyewale <oyewaleabdulsobur@gmail.com>
2424
* After a successful writing of the SQL script to fetch the required content from the fossology server, I proceeded to write a python program to embed the PostgreSQL script into the program using the psycog library to achieve the connection to the Postgres database server.
2525
* With this, i was able to automate the collection of copyright content data from the fossology server running in the local host.
2626

27+
![image](/img/data-pipeline/script1.png)
28+
![image](/img/data-pipeline/script2.png)
29+
2730

2831
## Meeting Discussion:
2932
* I discuss with the mentors about the progress of the week and how the project s going, including if there was any obstacle.

docs/2025/data-pipeline/updates/2025.06.25.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,10 @@ SPDX-FileCopyrightText: 2025 Abdulsobur Oyewale <oyewaleabdulsobur@gmail.com>
2525

2626
* I created a `pipeline.yml` file and applied the above script preprocessing and Data spliting script into the pipeline, and included the ability to download
2727
the output from each script from the logs while it's performing the triggered GitHub actions.
28+
![image](/img/data-pipeline/pipe.png)
29+
30+
![image](/img/data-pipeline/pipe1.png)
31+
2832
* I was able to deploy this into my GitHub repository to allow me to test this feature and changes separately on my own GitHub Actions before going ahead to create a Pull Request.
2933

3034
## Meeting Discussion:
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
---
2+
title: Week 6
3+
author: Abdulsobur Oyewale
4+
tags: [gsoc25, Data Pipeline for Safaa]
5+
---
6+
7+
<!--
8+
SPDX-License-Identifier: CC-BY-SA-4.0
9+
10+
SPDX-FileCopyrightText: 2025 Abdulsobur Oyewale <oyewaleabdulsobur@gmail.com>
11+
-->
12+
13+
# WEEK 5
14+
*(July 02, 2025)*
15+
16+
## Attendees:
17+
- [Kaushlendra Pratap](https://github.com/Kaushl2208)
18+
19+
### Engagements
20+
* This week was about migration of all current available progress so far in the demo repository into the main Safaa repository.
21+
It was really challenging and an eye-opener for me on working with the repository codebase. I faced challenges like incompatible code,
22+
trying to fit in the codes i have written so far into the repository, and a lot of git conflict.
23+
I was able to resolve all the code conflict, pushed it to one of my repository branch, and submitted a pull request for review.
24+
But not much of the git conflict resolved.
25+
26+
## Meeting Discussion:
27+
* In this week meeting, I had the opportunity to have [Kaushlendra Pratap](https://github.com/Kaushl2208) in the meeting. He worked me through resolving this git conflict,
28+
explain what was the cause, gave me guide on how to resolve the problem, explained every line of commands we used and what it does, and how this can
29+
be resolved if such issue comes up in the future.
30+
31+
![image](/img/data-pipeline/git_issues.png)
32+
![image](/img/data-pipeline/git_resolve.png)
33+
34+
## Subsequent Steps
35+
* After submitting my code and opening a pull request, i got a lot of feedback on where i can improve, and make adjustment to make the code much better.
36+
My main goal in the next week will be looking into this review, making corrections based on the feedbacks, and learning from it
361 KB
Loading
374 KB
Loading

static/img/data-pipeline/pipe.png

88.1 KB
Loading

static/img/data-pipeline/pipe1.png

70.8 KB
Loading
182 KB
Loading
189 KB
Loading

0 commit comments

Comments
 (0)