Skip to content

Commit 6217180

Browse files
authored
Update README.md
issue This PR Solves In microservices, logs are often collected using busy waiting loops, where the system constantly checks for new logs. This leads to several issues: Wasting Resources: The system keeps checking for new logs, using up CPU resources even when there are no logs to process. Increased Latency: Logs are delayed because the system is busy checking for new data rather than processing logs as they arrive. This PR solves these problems by switching to a more efficient event-driven log aggregation system. Instead of constantly checking for new logs, the system reacts only when new logs are available, reducing unnecessary CPU usage and improving scalability and performance.
1 parent b375919 commit 6217180

File tree

1 file changed

+105
-44
lines changed

1 file changed

+105
-44
lines changed

microservices-log-aggregation/README.md

Lines changed: 105 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -48,78 +48,139 @@ The `CentralLogStore` is responsible for storing the logs collected from various
4848

4949
```java
5050
public class CentralLogStore {
51+
private final List<LogEntry> logs = new ArrayList<>();
5152

52-
private final List<LogEntry> logs = new ArrayList<>();
53+
public void storeLog(LogEntry logEntry) {
54+
logs.add(logEntry);
55+
}
56+
public void displayLogs() {
57+
logs.forEach(System.out::println);
58+
}
59+
}
60+
```
61+
62+
The `LogAggregator` collects logs from various services and stores them in the `CentralLogStore`. It filters logs based on their log level.
63+
64+
65+
```java
66+
//This will be the structure to hold the log data
67+
public class LogEntry {
68+
private final String serviceName;
69+
private final String logLevel;
70+
private final String message;
71+
private final LocalDateTime timestamp;
72+
73+
public LogEntry(String serviceName, String logLevel, String message, LocalDateTime timestamp) {
74+
this.serviceName = serviceName;
75+
this.logLevel = logLevel;
76+
this.message = message;
77+
this.timestamp = timestamp;
78+
}
79+
80+
@Override
81+
public String toString() {
82+
return "LogEntry{" +
83+
"serviceName='" + serviceName + '\'' +
84+
", logLevel='" + logLevel + '\'' +
85+
", message='" + message + '\'' +
86+
", timestamp=" + timestamp +
87+
'}';
88+
}
89+
public String getServiceName() {
90+
return serviceName;
91+
}
92+
93+
public String getLogLevel() {
94+
return logLevel;
95+
}
5396

54-
public void storeLog(LogEntry logEntry) {
55-
logs.add(logEntry);
56-
}
97+
public String getMessage() {
98+
return message;
99+
}
57100

58-
public void displayLogs() {
59-
logs.forEach(System.out::println);
60-
}
101+
public LocalDateTime getTimestamp() {
102+
return timestamp;
103+
}
61104
}
62105
```
63106

64-
The `LogAggregator` collects logs from various services and stores them in the `CentralLogStore`. It filters logs based on their log level.
65107

66108
```java
67109
public class LogAggregator {
110+
private final CentralLogStore centralLogStore;
111+
private final BlockingQueue<LogEntry> logQueue;
68112

69-
private final CentralLogStore centralLogStore;
70-
private final LogLevel minimumLogLevel;
71-
72-
public LogAggregator(CentralLogStore centralLogStore, LogLevel minimumLogLevel) {
73-
this.centralLogStore = centralLogStore;
74-
this.minimumLogLevel = minimumLogLevel;
75-
}
113+
public LogAggregator(CentralLogStore centralLogStore, BlockingQueue<LogEntry> logQueue) {
114+
this.centralLogStore = centralLogStore;
115+
this.logQueue = logQueue;
116+
}
76117

77-
public void collectLog(LogEntry logEntry) {
78-
if (logEntry.getLogLevel().compareTo(minimumLogLevel) >= 0) {
79-
centralLogStore.storeLog(logEntry);
118+
public void startLogAggregation() {
119+
new Thread(() -> {
120+
try {
121+
while (true) {
122+
LogEntry log = logQueue.take();
123+
centralLogStore.storeLog(log);
124+
}
125+
} catch (InterruptedException e) {
126+
Thread.currentThread().interrupt();
127+
}
128+
}).start();
80129
}
81-
}
82130
}
83131
```
84132

85133
The `LogProducer` represents a service that generates logs. It sends the logs to the `LogAggregator`.
86134

87135
```java
88136
public class LogProducer {
137+
private final String serviceName;
138+
private final BlockingQueue<LogEntry> logQueue;
89139

90-
private final String serviceName;
91-
private final LogAggregator logAggregator;
92-
93-
public LogProducer(String serviceName, LogAggregator logAggregator) {
94-
this.serviceName = serviceName;
95-
this.logAggregator = logAggregator;
96-
}
140+
public LogProducer(String serviceName, BlockingQueue<LogEntry> logQueue) {
141+
this.serviceName = serviceName;
142+
this.logQueue = logQueue;
143+
}
97144

98-
public void generateLog(LogLevel logLevel, String message) {
99-
LogEntry logEntry = new LogEntry(serviceName, logLevel, message, LocalDateTime.now());
100-
logAggregator.collectLog(logEntry);
101-
}
145+
public void generateLog(String logLevel, String message) {
146+
LogEntry logEntry = new LogEntry(serviceName, logLevel, message, java.time.LocalDateTime.now());
147+
try {
148+
logQueue.put(logEntry);
149+
} catch (InterruptedException e) {
150+
Thread.currentThread().interrupt();
151+
}
152+
}
102153
}
103154
```
104155

105156
The `main` application creates services, generates logs, aggregates, and finally displays the logs.
106157

107158
```java
108159
public class App {
109-
110-
public static void main(String[] args) throws InterruptedException {
111-
final CentralLogStore centralLogStore = new CentralLogStore();
112-
final LogAggregator aggregator = new LogAggregator(centralLogStore, LogLevel.INFO);
113-
114-
final LogProducer serviceA = new LogProducer("ServiceA", aggregator);
115-
final LogProducer serviceB = new LogProducer("ServiceB", aggregator);
116-
117-
serviceA.generateLog(LogLevel.INFO, "This is an INFO log from ServiceA");
118-
serviceB.generateLog(LogLevel.ERROR, "This is an ERROR log from ServiceB");
119-
serviceA.generateLog(LogLevel.DEBUG, "This is a DEBUG log from ServiceA");
120-
121-
centralLogStore.displayLogs();
122-
}
160+
public static void main(String[] args) {
161+
CentralLogStore centralLogStore = new CentralLogStore();
162+
163+
BlockingQueue<LogEntry> logQueue = new LinkedBlockingQueue<>();
164+
165+
LogAggregator logAggregator = new LogAggregator(centralLogStore, logQueue);
166+
logAggregator.startLogAggregation(); // Start aggregation in the background
167+
168+
169+
LogProducer serviceA = new LogProducer("ServiceA", logQueue);
170+
LogProducer serviceB = new LogProducer("ServiceB", logQueue);
171+
172+
173+
serviceA.generateLog("INFO", "This is an INFO log from ServiceA");
174+
serviceB.generateLog("ERROR", "This is an ERROR log from ServiceB");
175+
serviceA.generateLog("DEBUG", "This is a DEBUG log from ServiceA");
176+
177+
try {
178+
Thread.sleep(2000); // Wait for logs to be processed
179+
} catch (InterruptedException e) {
180+
Thread.currentThread().interrupt();
181+
}
182+
centralLogStore.displayLogs();
183+
}
123184
}
124185
```
125186

0 commit comments

Comments
 (0)