Skip to content

Commit 05bb3b0

Browse files
Copilotgadicc
Copilot
and
gadicc
authored
chore(copilot): add comprehensive onboarding instructions (#943)
* Initial plan * Initialize repository exploration for Copilot instructions * Create comprehensive Copilot instructions with validated commands and timing * Enhance Copilot instructions with detailed validation scenarios and workflows * Finalize comprehensive Copilot instructions with complete module coverage and project insights * Update CI/CD documentation to reflect GitHub Actions migration from CircleCI * Update Copilot instructions to use optimized GitHub Actions setup --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: gadicc <381978+gadicc@users.noreply.github.com>
1 parent dd97942 commit 05bb3b0

14 files changed

Lines changed: 463 additions & 40 deletions

‎.github/copilot-instructions.md‎

Lines changed: 320 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,320 @@
1+
# Yahoo Finance 2 Development Instructions
2+
3+
Yahoo Finance 2 is a TypeScript/Deno library that provides programmatic access to Yahoo Finance data. It runs on Deno for development but builds to NPM packages for distribution. The library includes modules for stock quotes, historical data, financial summaries, search, and more.
4+
5+
Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here.
6+
7+
## Working Effectively
8+
9+
### Environment Setup
10+
- **PREFERRED**: Use the automated GitHub Actions setup for optimal performance and reliability:
11+
- Environment is automatically configured via `.github/workflows/copilot-setup-steps.yml`
12+
- Deno v2.x runtime installed with caching enabled
13+
- Dependencies pre-installed with optimized caching
14+
- Node.js v20 with npm/npx caching configured
15+
- No manual TLS CA store configuration needed
16+
17+
- **Manual Setup** (if GitHub Actions not available):
18+
```bash
19+
# Install Deno runtime
20+
wget https://github.com/denoland/deno/releases/latest/download/deno-x86_64-unknown-linux-gnu.zip
21+
unzip deno-x86_64-unknown-linux-gnu.zip
22+
chmod +x deno
23+
sudo mv deno /usr/local/bin/
24+
25+
# Install dependencies (much faster with caching)
26+
deno install
27+
```
28+
29+
### Optimized Development Process
30+
- **Dependencies**: With GitHub Actions setup, dependencies install in ~30-60 seconds (vs 5+ minutes manually)
31+
- **Build process**: NPM build completes in ~1 minute (vs 2+ minutes manually)
32+
- **Test execution**: Test suite runs in ~1-2 minutes with caching optimizations
33+
- **No TLS issues**: GitHub Actions environment resolves certificate and firewall issues automatically
34+
35+
### Core Development Commands
36+
- View available tasks: `deno task`
37+
- Run tests: `deno test -A --no-lock --parallel`
38+
- Build NPM package: `deno task build:npm`
39+
- Generate schemas: `deno task schema`
40+
- Run CLI tool: `deno task cli <module> <args>`
41+
- Lint code: `deno lint`
42+
- Format code: `deno fmt`
43+
44+
### Development Workflow
45+
1. **GitHub Actions Environment**: Preferred for optimal setup with caching and dependency management
46+
2. **ALWAYS** run schema generation after changing TypeScript interfaces: `deno task schema`
47+
3. **ALWAYS** run `deno fmt` and `deno lint` before committing changes
48+
4. Use `--no-lock` flag if encountering lockfile issues during development
49+
5. HTTP requests are cached in `tests/fixtures/http` (~450k lines of test data) - delete relevant files to refresh test data
50+
6. When adding new modules, follow the pattern: `.ts` file with `@yf-schema` comment + `.test.ts` + schema generation
51+
52+
## Validation
53+
54+
### Manual Testing Scenarios
55+
After making code changes, ALWAYS test the following scenarios to validate functionality:
56+
57+
#### CLI Testing (requires network access)
58+
- **Basic quote lookup**: `deno task cli quote AAPL`
59+
- **Module with options**: `deno task cli quoteSummary AAPL '{"modules":["price", "summaryDetail"]}'`
60+
- **Search functionality**: `deno task cli search AAPL`
61+
- **Historical data**: `deno task cli historical AAPL`
62+
- **Available modules**: `deno task cli --help` (shows: autoc, chart, dailyGainers, dailyLosers, fundamentalsTimeSeries, historical, insights, options, quote, quoteSummary, recommendationsBySymbol, screener, search, trendingSymbols)
63+
- **Help command**: `deno task cli --help`
64+
65+
#### Schema Generation Testing
66+
- **Regenerate schemas**: `deno task schema` (required after TypeScript interface changes)
67+
- **Verify schema files**: Check that `.schema.json` files are updated in `/src/modules/`
68+
- **Schema validation**: Look for `@yf-schema` comments in module files - only these are processed
69+
70+
#### Build and Code Quality Testing
71+
- **NPM build validation**: `deno task build:npm` (builds distributable package)
72+
- **Linting**: `deno lint` (expect some existing lint errors - focus on new code)
73+
- **Formatting**: `deno fmt --check` or `deno fmt` to auto-format
74+
- **Test execution**: `deno test -A --no-lock --parallel`
75+
76+
#### Development Workflow Testing
77+
1. Make a small TypeScript interface change in a module file
78+
2. Run schema generation: `deno task schema`
79+
3. Run tests to verify nothing broke: `deno test -A --no-lock --parallel`
80+
4. Format and lint: `deno fmt && deno lint`
81+
5. Test CLI functionality with the changed module
82+
83+
### Network and Certificate Issues
84+
- **GitHub Actions Environment**: TLS certificate and firewall issues automatically resolved
85+
- **Manual Environment**: If encountering SSL certificate errors, use `DENO_TLS_CA_STORE=system`
86+
- Network access required for:
87+
- Dependency downloads (JSR and NPM registries) - ~30-60 seconds with GitHub Actions caching
88+
- Yahoo Finance API calls during testing and CLI usage
89+
- Schema generation (requires npm package access)
90+
- **Cached test data**: ~450k lines of HTTP responses cached in `tests/fixtures/http/`
91+
- **Offline development**: Lint, format, and local file operations work without network access
92+
93+
## Common Issues and Solutions
94+
95+
### Build Failures
96+
- **GitHub Actions Environment**: Most SSL and network issues automatically resolved
97+
- **Manual Environment**: Use `DENO_TLS_CA_STORE=system` for SSL certificate issues
98+
- **Lockfile Corruption**: Use `--no-lock` flag to bypass lockfile issues
99+
- **Timeout Issues**: Much faster with GitHub Actions caching - builds typically complete in 1-2 minutes
100+
101+
### Network Dependencies
102+
- The project requires network access to NPM registry and JSR (JavaScript Registry)
103+
- **With GitHub Actions**: Dependencies install in ~30-60 seconds with caching
104+
- **Manual setup**: Initial dependency download: ~5 minutes (use appropriate timeouts)
105+
- Cached dependencies significantly reduce subsequent command execution times
106+
107+
### Testing
108+
- Tests use cached HTTP responses stored in `tests/fixtures/http`
109+
- To refresh test data, delete relevant fixture files
110+
- Tests run in parallel by default for faster execution
111+
- Some tests may require actual Yahoo Finance API access
112+
113+
## Project Structure
114+
115+
### Key Directories
116+
- `/src` - Main TypeScript source code
117+
- `/src/modules` - Individual Yahoo Finance API modules (quote, chart, search, etc.)
118+
- `/src/lib` - Core library functionality
119+
- `/tests` - Test files and HTTP fixtures
120+
- `/bin` - CLI entry point
121+
- `/scripts` - Build and schema generation scripts
122+
- `/docs` - Documentation
123+
- `deno.json` - Deno configuration with tasks and dependencies
124+
125+
### Important Files
126+
- `deno.json` - Project configuration and task definitions
127+
- `deno.lock` - Dependency lockfile (may need `--no-lock` to bypass)
128+
- `CONTRIBUTING.md` - Additional development guidance
129+
- `/scripts/build_npm.ts` - NPM package build script
130+
- `/scripts/schema-gen.ts` - TypeScript to JSON schema generator
131+
132+
## Module Development
133+
134+
### Adding a New Module
135+
1. Create module file: `src/modules/myModule.ts`
136+
2. Add TypeScript interfaces with `@yf-schema` comment (required for schema generation)
137+
3. Create test file: `src/modules/myModule.test.ts`
138+
4. Run `DENO_TLS_CA_STORE=system deno task schema` to generate `myModule.schema.json`
139+
5. Add module to `src/index-common.ts` for export
140+
6. Create documentation in `docs/modules/myModule.md`
141+
7. Update README.md to link new module documentation
142+
8. Test via CLI: `deno task cli myModule <symbol> <options>`
143+
144+
### Schema Generation Details
145+
- **CRITICAL**: Only files with `@yf-schema` keyword are processed by schema generator
146+
- **CRITICAL**: Must run `deno task schema` after any interface changes
147+
- Schema files are automatically generated as `*.schema.json` alongside `*.ts` files
148+
- Schemas enable runtime validation of Yahoo Finance API responses
149+
- Pattern: `// @yf-schema: see the docs on how this file is automatically updated.`
150+
151+
### Testing New Code
152+
- **Unit tests**: Use existing test patterns with cached HTTP responses
153+
- **Integration tests**: Test CLI commands with real Yahoo Finance data
154+
- **Schema validation**: Verify interfaces match actual API responses
155+
- **Cache management**: Delete specific files in `tests/fixtures/http/` to refresh data for your tests
156+
157+
## CI/CD Pipeline
158+
159+
### GitHub Actions Workflow (`.github/workflows/test-release.yaml`)
160+
1. **Environment Setup** (`.github/actions/setup` composite action):
161+
- Install Deno v2.x runtime with caching enabled
162+
- Install dependencies (`deno install`) - takes ~5 minutes
163+
- Setup Node.js v20 with npm cache optimization
164+
2. **Testing**: Run tests with coverage (`deno task test --coverage`) - takes ~3 minutes
165+
3. **Coverage Processing**: Generate LCOV report (`deno coverage --lcov ./coverage > coverage.lcov`)
166+
4. **Build**: Build NPM package (`deno task build:npm`) - takes ~2 minutes
167+
5. **Release**: Run semantic-release for publishing to both NPM and JSR (JavaScript Registry)
168+
169+
**Notes**:
170+
- Coverage is currently generated but not yet uploaded to codecov
171+
- Test results are not yet used for repository badges
172+
- JSR publishing has been added alongside NPM releases
173+
174+
### Expected Timings
175+
**With GitHub Actions Setup (Recommended):**
176+
- **Dependency installation**: ~30-60 seconds (with caching)
177+
- **Test execution**: ~1-2 minutes (optimized with caching)
178+
- **NPM build**: ~1 minute (cached dependencies)
179+
- **Schema generation**: ~30 seconds
180+
- **Linting/formatting**: ~10 seconds
181+
182+
**Manual Setup (Fallback):**
183+
- **Dependency installation**: ~5+ minutes (NEVER CANCEL)
184+
- **Test execution**: ~2-3 minutes (NEVER CANCEL)
185+
- **NPM build**: ~2+ minutes (NEVER CANCEL)
186+
- **Schema generation**: ~30 seconds
187+
- **Linting/formatting**: ~10 seconds
188+
189+
## Environment Variables
190+
191+
### Network Operations
192+
- **GitHub Actions Environment**: No manual environment variables needed
193+
- **Manual Environment**: `DENO_TLS_CA_STORE=system` - Fixes SSL certificate issues with npm registry
194+
- `YF_QUERY_HOST` - Yahoo Finance API host (defaults to query2.yahoo.finance.com)
195+
196+
### Development Flags
197+
- `FETCH_DEVEL=nocache` - Disable HTTP caching for development
198+
- `NODE_ENV=test` - Enable strict validation mode
199+
200+
## Troubleshooting
201+
202+
### Common Error Messages
203+
**GitHub Actions Environment:** Most common network/TLS issues are automatically resolved.
204+
205+
**Manual Environment:**
206+
- "Failed loading https://registry.npmjs.org/" - Use `DENO_TLS_CA_STORE=system`
207+
- "invalid peer certificate: UnknownIssuer" - SSL certificate issue, use system CA store
208+
- "Failed upgrading lockfile" - Use `--no-lock` flag
209+
- "JSR package manifest failed to load" - Network connectivity issue to JSR registry
210+
211+
## Example Development Workflow
212+
213+
### Complete Example: Adding a Simple Interface Change
214+
215+
**With GitHub Actions Setup (Recommended):**
216+
```bash
217+
# 1. Environment automatically configured via copilot-setup-steps.yml
218+
# No manual setup needed
219+
220+
# 2. Make a change to a TypeScript interface in src/modules/quote.ts
221+
# (example: add a new optional field to QuoteBase interface)
222+
223+
# 3. Regenerate schemas (REQUIRED after interface changes)
224+
deno task schema # Takes ~30 seconds
225+
226+
# 4. Run tests to ensure nothing broke
227+
deno test -A --no-lock --parallel # Takes ~1-2 minutes with caching
228+
229+
# 5. Test the specific module via CLI
230+
deno task cli quote AAPL # Verify real API calls work
231+
232+
# 6. Format and lint code
233+
deno fmt # Auto-formats files
234+
deno lint # Shows any linting issues
235+
236+
# 7. Build NPM package to verify distribution works
237+
deno task build:npm # Takes ~1 minute with caching
238+
```
239+
240+
**Manual Setup (Fallback):**
241+
```bash
242+
# 1. Set up environment manually
243+
export DENO_TLS_CA_STORE=system
244+
245+
# 2. Make a change to a TypeScript interface in src/modules/quote.ts
246+
# (example: add a new optional field to QuoteBase interface)
247+
248+
# 3. Regenerate schemas (REQUIRED after interface changes)
249+
deno task schema # Takes ~30 seconds
250+
251+
# 4. Run tests to ensure nothing broke
252+
deno test -A --no-lock --parallel # Takes 2-3 minutes. NEVER CANCEL.
253+
254+
# 5. Test the specific module via CLI
255+
deno task cli quote AAPL # Verify real API calls work
256+
257+
# 6. Format and lint code
258+
deno fmt # Auto-formats files
259+
deno lint # Shows any linting issues
260+
261+
# 7. Build NPM package to verify distribution works
262+
deno task build:npm # Takes ~2 minutes. NEVER CANCEL.
263+
```
264+
265+
### Example: Refreshing Test Data for a Module
266+
```bash
267+
# 1. Delete cached HTTP responses for specific API calls
268+
rm tests/fixtures/http/quote-AAPL.json
269+
rm tests/fixtures/http/quote-TSLA.json
270+
271+
# 2. Run tests - they will fetch fresh data and cache it
272+
deno test -A --no-lock src/modules/quote.test.ts
273+
274+
# 3. Verify new cached data looks correct
275+
cat tests/fixtures/http/quote-AAPL.json | head -20
276+
```
277+
278+
### Performance Notes
279+
**With GitHub Actions Setup:**
280+
- Environment setup in ~30-60 seconds with comprehensive caching
281+
- Subsequent runs much faster due to optimized dependency and npm caching
282+
- Use `--parallel` flag for tests to maximize performance
283+
- Build artifacts are generated in `/npm` directory for NPM distribution
284+
285+
**Manual Setup:**
286+
- First-time setup requires significant network downloading (~5+ minutes)
287+
- Subsequent runs are much faster due to caching
288+
- Use `--parallel` flag for tests to maximize performance
289+
- Build artifacts are generated in `/npm` directory for NPM distribution
290+
291+
## Key Project Insights
292+
293+
### Architecture Overview
294+
- **Runtime**: Deno for development, compiles to Node.js/NPM for distribution
295+
- **API Coverage**: Comprehensive Yahoo Finance API access (quotes, historical data, search, financials, etc.)
296+
- **Type Safety**: Full TypeScript with runtime validation via JSON schemas
297+
- **Caching**: HTTP responses cached to disk for consistent/fast testing
298+
- **CLI**: Full command-line interface for all modules and functions
299+
300+
### Development Philosophy
301+
- **Schema-driven**: TypeScript interfaces with `@yf-schema` generate runtime validation
302+
- **Test-first**: Extensive cached test data ensures consistent behavior
303+
- **Performance-conscious**: Parallel testing, efficient caching, minimal dependencies
304+
- **Cross-platform**: Develop on Deno, distribute via NPM for broad compatibility
305+
306+
### Common Modules and Use Cases
307+
- **quote**: Get current stock prices and basic info
308+
- **quoteSummary**: Detailed financial data with submodules (earnings, balance sheet, etc.)
309+
- **historical**: Historical price data with date ranges
310+
- **search**: Find stocks/securities by symbol or name
311+
- **chart**: Price charts with various timeframes and indicators
312+
- **trendingSymbols**: Currently trending stocks by region
313+
- **options**: Options chain data for stocks
314+
- **insights**: Market insights and analyst recommendations
315+
316+
## Legacy Documentation
317+
The project includes legacy documentation for Version 2 which used Node.js/yarn. Current development uses Deno exclusively. Ignore references to:
318+
- `yarn` commands (use `deno task` instead)
319+
- `npm` commands for development (use `deno` commands)
320+
- TypeScript compilation with `tsc` (handled by Deno)

‎deno-x86_64-unknown-linux-gnu.zip‎

42.4 MB
Binary file not shown.

0 commit comments

Comments
 (0)