@@ -316,90 +316,35 @@ Logging is configured in `log4js.json`. Logs are written to both console and `lo
316316
317317## Development
318318
319- ### Prerequisites
319+ For contributors, comprehensive documentation is available in the [ Contributor Guide ] ( guides/contributor/ ) :
320320
321- - Node.js 20+ (CI uses Node 23)
322- - Docker (for running Spark Connect server)
323- - npm
321+ - ** [ Getting Started] ( guides/contributor/GETTING_STARTED.md ) ** - Set up your development environment
322+ - ** [ Code Style Guide] ( guides/contributor/CODE_STYLE.md ) ** - Coding conventions and best practices
323+ - ** [ Build and Test] ( guides/contributor/BUILD_AND_TEST.md ) ** - Building, testing, and running the project
324+ - ** [ IDE Setup] ( guides/contributor/IDE_SETUP.md ) ** - Recommended IDE configurations
325+ - ** [ Submitting Changes] ( guides/contributor/SUBMITTING_CHANGES.md ) ** - How to submit pull requests
324326
325- ### Setup
326-
327- 1 . ** Clone the repository** :
328- ``` bash
329- git clone https://github.com/yaooqinn/spark.js.git
330- cd spark.js
331- ```
332-
333- 2 . ** Install dependencies** :
334- ``` bash
335- npm install
336- ```
337-
338- 3 . ** Start Spark Connect server** (required for tests):
339- ``` bash
340- # Build the Docker image (first time only, ~20-30 seconds)
341- docker build -t scs .github/docker
342-
343- # Start the server
344- docker run --name sparkconnect -p 15002:15002 -d scs
345-
346- # Wait 15-20 seconds for server to be ready
347- # Check logs to confirm it's running:
348- docker logs sparkconnect
349- # Look for: "Spark Connect server started at: [::]:15002"
350- ```
351-
352- 4 . ** Stop the server** when done:
353- ``` bash
354- docker stop sparkconnect
355- docker rm sparkconnect
356- ```
357-
358- ### Running Tests
327+ ### Quick Start for Contributors
359328
360329``` bash
361- # Make sure Spark Connect server is running first!
362- npm test
363- ```
364-
365- The test suite includes:
366- - 28 test files
367- - 179+ test cases
368- - Tests for DataFrame operations, SQL, readers/writers, catalog, types, etc.
369-
370- Tests require a running Spark Connect server on port 15002.
330+ # Clone and install
331+ git clone https://github.com/yaooqinn/spark.js.git
332+ cd spark.js
333+ npm install
371334
372- ### Linting
335+ # Start Spark Connect server for testing
336+ docker build -t scs .github/docker
337+ docker run --name sparkconnect -p 15002:15002 -d scs
373338
374- ``` bash
339+ # Run tests and linting
340+ npm test
375341npm run lint
376- ```
377-
378- ESLint is configured with TypeScript support. Generated protobuf code in ` src/gen/ ` is ignored.
379-
380- ### Building
381342
382- Compile TypeScript to JavaScript:
383-
384- ``` bash
385- npx tsc --build
386- ```
387-
388- Type check without emitting files:
389-
390- ``` bash
391- npx tsc --noEmit
392- ```
393-
394- ### Protobuf Code Generation
395-
396- If you modify ` .proto ` files in ` protobuf/ ` :
397-
398- ``` bash
399- npx buf generate
343+ # Clean up
344+ docker stop sparkconnect && docker rm sparkconnect
400345```
401346
402- Generated TypeScript code is placed in ` src/gen/ ` .
347+ For detailed instructions, see the [ Contributor Guide ] ( guides/contributor/ ) .
403348
404349### Project Structure
405350
@@ -452,70 +397,22 @@ npx ts-node example/org/apache/spark/sql/example/Pi.ts
452397
453398## Contributing
454399
455- Contributions are welcome! Here's how to contribute:
456-
457- 1 . ** Fork the repository** and create a feature branch
458- 2 . ** Make your changes** following the existing code style
459- 3 . ** Add tests** for new functionality
460- 4 . ** Run linting** : ` npm run lint `
461- 5 . ** Run tests** : ` npm test ` (requires Spark Connect server)
462- 6 . ** Submit a pull request** with a clear description
463-
464- ### Code Style
465-
466- - Use 2 spaces for indentation
467- - Follow TypeScript best practices
468- - Add Apache License header to new files
469- - Write descriptive commit messages
470-
471- ### Running CI Checks Locally
472-
473- Before submitting a PR, ensure CI checks pass:
474-
475- ``` bash
476- # Linting
477- npm run lint
400+ Contributions are welcome! Please read the [ Contributor Guide] ( guides/contributor/ ) for detailed information on:
478401
479- # Tests (requires Docker)
480- docker build -t scs .github/docker
481- docker run --name sparkconnect -p 15002:15002 -d scs
482- npm test
483- docker stop sparkconnect && docker rm sparkconnect
484- ```
402+ - Setting up your development environment
403+ - Code style and conventions
404+ - Building and testing
405+ - Submitting pull requests
485406
486- ### Publishing to npm
407+ ### Quick Contributing Guide
487408
488- To publish a new version to npm (maintainers only):
489-
490- 1 . ** Update the version** in ` package.json ` :
491- ``` bash
492- npm version patch # or minor, or major
493- ```
494-
495- 2 . ** Build the package** :
496- ``` bash
497- npm run build
498- ```
499-
500- 3 . ** Test the package locally** (optional):
501- ``` bash
502- npm pack
503- # This creates a .tgz file you can test with: npm install spark.js-0.1.0.tgz
504- ```
505-
506- 4 . ** Publish to npm** :
507- ``` bash
508- npm login # Login to npm (first time only)
509- npm publish
510- ```
511-
512- 5 . ** Tag the release** on GitHub:
513- ``` bash
514- git tag v0.1.0
515- git push origin v0.1.0
516- ```
409+ 1 . ** Fork** the repository and create a feature branch
410+ 2 . ** Follow** the [ Code Style Guide] ( guides/contributor/CODE_STYLE.md )
411+ 3 . ** Add tests** for new functionality
412+ 4 . ** Run checks** : ` npm run lint ` and ` npm test `
413+ 5 . ** Submit** a pull request with a clear description
517414
518- ** Note ** : The ` prepublishOnly ` script automatically runs the build before publishing .
415+ See [ Submitting Changes ] ( guides/contributor/SUBMITTING_CHANGES.md ) for detailed instructions .
519416
520417## Roadmap
521418
0 commit comments