Skip to content

Commit 4b890ed

Browse files
marevolclaude
marevol
and
claude
authored
Initialize new project setup (#2945)
* Add CLAUDE.md file for Claude Code guidance This file provides comprehensive guidance for future Claude Code instances working in this repository, including: - Development commands (build, test, run, code generation) - High-level architecture overview (LastaFlute, DBFlute, OpenSearch) - Key package structure and application layers - Important configuration files and their purposes - Common development patterns for adding features - Testing patterns and debugging information The file focuses on "big picture" architecture that requires reading multiple files to understand, avoiding repetition of README content and obvious instructions. * test: Pin fess-testdata to specific commit in integration tests Pin fess-testdata repository to commit f19176ab1b7ddc0a40393a8cbbb8d1c17b27c3ce to ensure test stability and prevent test failures due to changes in test data. This fixes integration test failures in SearchApiTests where newer test data files did not contain expected keywords (e.g., test.pub missing "java" and "public"). --------- Co-authored-by: Claude <[email protected]>
1 parent ad598dc commit 4b890ed

File tree

2 files changed

+342
-0
lines changed

2 files changed

+342
-0
lines changed

‎CLAUDE.md‎

Lines changed: 339 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,339 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Project Overview
6+
7+
Fess is an Enterprise Search Server built on OpenSearch. It's a Java-based web application that crawls and indexes documents from various sources (web, file systems, databases, cloud storage) and provides full-text search capabilities through both a user-facing search UI and an administrative interface.
8+
9+
## Development Commands
10+
11+
### Setup
12+
```bash
13+
# Download OpenSearch plugins (required before first build)
14+
mvn antrun:run
15+
16+
# Generate DBFlute source code (when schema changes)
17+
mvn dbflute:download # One-time setup
18+
mvn dbflute:freegen
19+
mvn license:format
20+
```
21+
22+
### Build
23+
```bash
24+
# Standard build (compiles, runs unit tests, packages WAR)
25+
mvn package
26+
27+
# Build distribution packages
28+
mvn package # Creates .zip in target/releases/
29+
mvn rpm:rpm # Creates .rpm package
30+
mvn jdeb:jdeb # Creates .deb package
31+
32+
# Clean build
33+
mvn clean package
34+
```
35+
36+
### Testing
37+
```bash
38+
# Run unit tests only (*Test.java files)
39+
mvn test
40+
41+
# Run integration tests (*Tests.java files in src/test/java/org/codelibs/fess/it/)
42+
# Note: Requires Fess server and OpenSearch to be running
43+
mvn test -P integrationTests -Dtest.fess.url="http://localhost:8080" -Dtest.search_engine.url="http://localhost:9201"
44+
45+
# Run a single integration test
46+
mvn test -P integrationTests -Dtest.fess.url="http://localhost:8080" -Dtest.search_engine.url="http://localhost:9201" -Dtest=SearchApiTests
47+
48+
# Run a single unit test
49+
mvn test -Dtest=SearchEngineClientTest
50+
```
51+
52+
### Running the Application
53+
```bash
54+
# Run from IDE: Execute main method in org.codelibs.fess.FessBoot
55+
# Access at: http://localhost:8080/
56+
57+
# Run from command line (after building)
58+
./bin/fess
59+
60+
# In IDE, set VM options if needed:
61+
# -Dfess.log.path=/path/to/logs
62+
# -Dfess.var.path=/path/to/var
63+
```
64+
65+
### Code Formatting
66+
```bash
67+
# Format code according to project standards
68+
mvn formatter:format
69+
70+
# Validate license headers
71+
mvn license:check
72+
73+
# Add license headers
74+
mvn license:format
75+
```
76+
77+
## Architecture Overview
78+
79+
### Tech Stack
80+
- **Web Framework**: LastaFlute (Japanese MVC framework, convention over configuration)
81+
- **DI Container**: Lasta Di (lightweight dependency injection)
82+
- **Data Access**: DBFlute (type-safe ORM-like framework, used with OpenSearch indices)
83+
- **Search Engine**: OpenSearch (stores both search documents and application data)
84+
- **App Server**: Embedded Tomcat (via tomcat-boot library)
85+
- **Crawler**: Custom fess-crawler library
86+
- **Scheduler**: Lasta Job (for crawl jobs and maintenance tasks)
87+
88+
### Application Layers
89+
90+
```
91+
┌─────────────────────────────────────────┐
92+
│ Presentation (app.web.* actions) │ JSP views, REST APIs
93+
├─────────────────────────────────────────┤
94+
│ Service Layer (app.service.*) │ Business logic
95+
├─────────────────────────────────────────┤
96+
│ Helper Layer (helper.*) │ Cross-cutting utilities
97+
├─────────────────────────────────────────┤
98+
│ Data Access (opensearch.*.exbhv) │ Repository pattern via DBFlute
99+
├─────────────────────────────────────────┤
100+
│ OpenSearch (3 indices) │ config, log, user indices
101+
└─────────────────────────────────────────┘
102+
```
103+
104+
### Key Package Structure
105+
106+
**src/main/java/org/codelibs/fess/**
107+
108+
- **FessBoot.java** - Application entry point (extends TomcatBoot)
109+
- **app/** - Application layer
110+
- **web/** - User-facing controllers (search, login, profile)
111+
- **web/admin/** - ~40 admin controllers (webconfig, fileconfig, scheduler, etc.)
112+
- **service/** - Business logic services (~50+ services)
113+
- **job/** - Background job definitions
114+
- **pager/** - Pagination utilities
115+
- **api/** - REST API layer
116+
- **engine/** - Search engine API managers
117+
- **json/** - JSON API endpoints
118+
- **opensearch/** - OpenSearch integration (DBFlute-generated + custom code)
119+
- **config/** - Configuration index (crawl configs, schedules, dictionaries, etc.)
120+
- **log/** - Log index (search logs, job logs)
121+
- **user/** - User index (users, groups, roles)
122+
- **client/** - Search client implementations
123+
- **query/** - Custom query builders
124+
- **exbhv/** - Extended behavior classes (repositories)
125+
- **exentity/** - Extended entity classes (domain models)
126+
- **helper/** - Cross-cutting helpers (~30+ helpers)
127+
- IndexingHelper, CrawlingConfigHelper, DocumentHelper, etc.
128+
- **crawler/** - Web crawling engine
129+
- **processor/** - Content processors
130+
- **transformer/** - Data transformers
131+
- **service/** - Crawler services
132+
- **entity/** - Core domain entities
133+
- **exception/** - Custom exception hierarchy
134+
- **auth/** - Authentication framework
135+
- **chain/** - Authentication chain pattern
136+
- **sso/** - Single Sign-On implementations (AAD, OIDC, SAML, SPNEGO)
137+
- **ds/** - Pluggable Data Store connectors
138+
- **dict/** - Dictionary management
139+
- **mylasta/** - LastaFlute framework configuration
140+
- **util/** - Utility classes
141+
142+
### OpenSearch Indices Architecture
143+
144+
Fess uses OpenSearch for **both** search document storage and application data storage. There are three main index types:
145+
146+
1. **config index** (`opensearch.config.*`) - Application configuration
147+
- Crawl configurations (web, file, data store)
148+
- Schedules, dictionaries, labels, roles, etc.
149+
150+
2. **log index** (`opensearch.log.*`) - Application logs
151+
- Search logs, job logs, user info logs
152+
153+
3. **user index** (`opensearch.user.*`) - User management
154+
- Users, groups, roles
155+
156+
These indices use DBFlute's code generation to create type-safe entity and behavior classes, similar to traditional ORM but for OpenSearch.
157+
158+
### LastaFlute Action Pattern
159+
160+
Controllers are called "Actions" and follow these conventions:
161+
162+
```java
163+
public class ExampleAction extends FessSearchAction {
164+
@Resource
165+
private ExampleService exampleService;
166+
167+
@Execute
168+
public HtmlResponse index(ExampleForm form) {
169+
validate(form, messages -> {}, () -> {
170+
return asHtml(path_IndexJsp);
171+
});
172+
173+
exampleService.doSomething(form);
174+
175+
return asHtml(path_IndexJsp)
176+
.renderWith(data -> {
177+
data.register("result", exampleService.getData());
178+
});
179+
}
180+
}
181+
```
182+
183+
- Actions are in `app.web.*` or `app.web.admin.*`
184+
- Methods annotated with `@Execute` are web endpoints
185+
- Forms are validated using LastaFlute's validation framework
186+
- `HtmlResponse` for JSP rendering, `JsonResponse` for APIs
187+
- DI via `@Resource` annotation
188+
189+
### Service Pattern
190+
191+
Services contain business logic and are injected into Actions:
192+
193+
```java
194+
public class ExampleService {
195+
@Resource
196+
private ExampleBhv exampleBhv; // DBFlute behavior (repository)
197+
198+
public void doSomething() {
199+
// Use behavior classes to access OpenSearch
200+
exampleBhv.selectEntity(cb -> {
201+
cb.query().setId_Equal(id);
202+
}).ifPresent(entity -> {
203+
// Process entity
204+
});
205+
}
206+
}
207+
```
208+
209+
### Helper Pattern
210+
211+
Helpers provide reusable, cross-cutting functionality:
212+
- Stateless utility classes with complex logic
213+
- Injected via DI
214+
- Examples: `IndexingHelper`, `CrawlingConfigHelper`, `PermissionHelper`
215+
216+
### Dependency Injection
217+
218+
- Configuration files: `src/main/resources/*.xml` (app.xml, fess.xml, etc.)
219+
- Components auto-discovered via package scanning
220+
- Access via `@Resource` injection or `ComponentUtil.getComponent()`
221+
- Static access pattern: `ComponentUtil.getFooHelper()` for framework components
222+
223+
## Important Configuration Files
224+
225+
### Application Configuration
226+
- **src/main/resources/fess_config.properties** - Main application config (search engine URL, cipher keys, crawl settings)
227+
- **src/main/resources/fess_env_*.properties** - Environment-specific configs (web, crawler, suggest, thumbnail)
228+
- **src/main/resources/lasta_di.properties** - DI container settings
229+
230+
### DI Container (XML)
231+
- **src/main/resources/app.xml** - Main DI configuration
232+
- **src/main/resources/fess.xml** - Fess-specific components
233+
- **src/main/resources/fess_*.xml** - Modular configs (api, cors, dict, job, ldap, sso)
234+
- **src/main/resources/crawler_*.xml** - Crawler components
235+
- **src/main/resources/esclient.xml** - OpenSearch client configuration
236+
237+
### Localization
238+
- **src/main/resources/fess_label_*.properties** - UI labels (17 languages: en, ja, ar, cs, de, es, fr, hu, id, it, ko, nl, pl, pt, ro, ru, th, tr, zh_CN, zh_TW)
239+
- **src/main/resources/fess_message_*.properties** - Validation/error messages
240+
241+
### Views
242+
- **src/main/webapp/WEB-INF/view/** - JSP templates
243+
- **admin/** - Admin UI views
244+
- **search.jsp** - User search UI
245+
- **common/** - Shared templates
246+
247+
### OpenSearch
248+
- **src/main/resources/fess_indices/** - Index mapping definitions (JSON)
249+
- **src/main/resources/tika.xml** - Apache Tika configuration for content extraction
250+
251+
### Build & Deployment
252+
- **pom.xml** - Maven dependencies and plugins
253+
- **module.xml** - OpenSearch module management (Ant script)
254+
- **plugin.xml** - OpenSearch plugin management (Ant script)
255+
- **deps.xml** - Additional dependency management (Ant script)
256+
257+
## DBFlute Code Generation
258+
259+
DBFlute is used to generate type-safe data access code for OpenSearch indices:
260+
261+
```bash
262+
# Generate entity and behavior classes
263+
mvn dbflute:freegen
264+
```
265+
266+
Generated code locations:
267+
- **opensearch/{index}/bsentity/** - Base entities (do not edit)
268+
- **opensearch/{index}/exentity/** - Extended entities (customize here)
269+
- **opensearch/{index}/bsbhv/** - Base behaviors (do not edit)
270+
- **opensearch/{index}/exbhv/** - Extended behaviors (customize here)
271+
- **opensearch/{index}/cbean/** - Condition beans for queries
272+
273+
Configuration: `dbflute_fess/dfprop/`
274+
275+
## Testing Patterns
276+
277+
### Unit Tests
278+
- File pattern: `*Test.java`
279+
- Located in `src/test/java/` mirroring `src/main/java/`
280+
- Run with: `mvn test`
281+
- Use JUnit 5 (Jupiter) or JUnit 4
282+
- May extend `UnitFessTestCase` for LastaFlute support
283+
284+
### Integration Tests
285+
- File pattern: `*Tests.java` in `src/test/java/org/codelibs/fess/it/`
286+
- Require running Fess and OpenSearch instances
287+
- Use REST Assured for API testing
288+
- Run with: `mvn test -P integrationTests -Dtest.fess.url="http://localhost:8080" -Dtest.search_engine.url="http://localhost:9201"`
289+
290+
## Common Development Patterns
291+
292+
### Adding a New Admin Feature
293+
294+
1. Create entity in appropriate OpenSearch index (`opensearch.config.exentity`)
295+
2. Create behavior class if needed (`opensearch.config.exbhv`)
296+
3. Create service in `app.service`
297+
4. Create action in `app.web.admin`
298+
5. Create form beans (request/response)
299+
6. Create JSP view in `src/main/webapp/WEB-INF/view/admin/`
300+
7. Add labels to `fess_label_en.properties` (and other languages)
301+
8. Add messages to `fess_message_en.properties`
302+
303+
### Adding a New Crawler/Data Source
304+
305+
1. Implement data store connector in `ds` package
306+
2. Register in `DataStoreFactory`
307+
3. Add configuration entity/behavior
308+
4. Create admin UI (action + view)
309+
310+
### Adding Authentication Method
311+
312+
1. Implement in `sso` package
313+
2. Add to authentication chain in `auth.chain`
314+
3. Configure in `fess_sso.xml`
315+
316+
## Development Notes
317+
318+
- **Code Style**: Maven formatter plugin enforces style (`mvn formatter:format`)
319+
- **License Headers**: Apache 2.0 headers required (`mvn license:format`)
320+
- **Logging**: Log4j2 (`src/main/resources/log4j2.xml`)
321+
- **Internationalization**: All user-facing strings must be in `fess_label_*.properties`
322+
- **Validation**: Use LastaFlute's validation annotations on form beans
323+
- **Transaction Handling**: OpenSearch operations are generally non-transactional; use try-catch for error handling
324+
325+
## Debugging
326+
327+
- **Main Class**: `org.codelibs.fess.FessBoot`
328+
- **Default Port**: 8080 (configurable in tomcat_config.properties or system property `fess.port`)
329+
- **Log Location**: `target/fess-logs/` (development) or `/var/log/fess/` (production)
330+
- **Data Location**: `target/fess-var/` (development)
331+
- **Admin Login**: Default username/password is `admin/admin`
332+
333+
## External Dependencies
334+
335+
Fess integrates with external OpenSearch modules and plugins downloaded via `mvn antrun:run`:
336+
- Modules stored in: `modules/`
337+
- Plugins stored in: `plugins/`
338+
339+
These are managed by `module.xml` and `plugin.xml` Ant scripts.

‎src/test/resources/before_script.sh‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,9 @@ done
2828

2929
pushd /tmp >/dev/null
3030
git clone https://github.com/codelibs/fess-testdata.git
31+
cd fess-testdata
32+
git checkout f19176ab1b7ddc0a40393a8cbbb8d1c17b27c3ce
33+
cd ..
3134
popd >/dev/null
3235

3336
cat ${temp_log_file} ./fess-*/logs/*.log

0 commit comments

Comments
 (0)