Skip to content

Commit 0150b95

Browse files
markzegarellimarkzegarelli
committed
updates
1 parent 3c0f329 commit 0150b95

File tree

3 files changed

+227
-133
lines changed

3 files changed

+227
-133
lines changed

‎README.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,10 +23,14 @@ If you see an error or something incorrect on a specific page, click Request doc
2323

2424
### Pull Requests
2525

26-
If you would like to update the documentation yourself, we ask that you create a Pull Request to bring it to the team’s attention. Click Edit this page to edit the page directly on GitHub. Alternatively, you can clone the repository and submit your Pull Request that way.
26+
If you would like to update the documentation yourself, create a Pull Request to bring it to the team’s attention. Click Edit this page to edit the page directly on GitHub. Alternatively, you can clone the repository and submit your Pull Request that way. For more information, see Contributing and the Developer Guide
2727

2828
For more information about contributing, see Contributing.
2929

3030
## List of READMEs
3131

32-
- Contributing
32+
- [Contributing](#)
33+
- [Style Guide](#)
34+
- [Developer Guide](#)
35+
36+
## License

‎devguide-old.md

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
# Segment Docs Dev guide
2+
3+
The contents of this guide will help you get up and running with the Segment Docs local environment.
4+
5+
<!-- vscode-markdown-toc -->
6+
* 1. [Local development with `ruby` and `node`, without Config API](#LocaldevelopmentwithrubyandnodewithoutConfigAPI)
7+
* 1. [Changing a DevCenter Destination's name](#ChangingaDevCenterDestinationsname)
8+
* 2. [All about the Catalog script](#AllabouttheCatalogscript)
9+
* 2.1. [Connection Modes in the Catalog script](#ConnectionModesintheCatalogscript)
10+
* 3. [Developer information](#Developerinformation)
11+
* 3.1. [Layouts](#Layouts)
12+
* 3.2. [Config API + Catalog](#ConfigAPICatalog)
13+
* 3.2.1. [API Key](#APIKey)
14+
* 3.2.2. [Catalog Data + Doc Links](#CatalogDataDocLinks)
15+
* 3.2.3. [Object Sources and Warehouses](#ObjectSourcesandWarehouses)
16+
* 3.3. [Sidenav](#Sidenav)
17+
* 3.4. [Breadcrumb](#Breadcrumb)
18+
* 3.5. [Searching](#Searching)
19+
* 4. [Testing](#Testing)
20+
* 4.1. [Build Testing](#BuildTesting)
21+
* 4.2. [Manual Testing](#ManualTesting)
22+
23+
<!-- vscode-markdown-toc-config
24+
numbering=true
25+
autoSave=true
26+
/vscode-markdown-toc-config -->
27+
<!-- /vscode-markdown-toc -->
28+
29+
30+
31+
32+
### 1. <a name='LocaldevelopmentwithrubyandnodewithoutConfigAPI'></a>Local development with `ruby` and `node`, without Config API
33+
34+
If using OSX:
35+
* Install command line tools, `xcode-select --install`
36+
* Install `Ruby` >= 2.3.0 https://www.ruby-lang.org/en/documentation/installation/
37+
* Ensure you're running `RubyGems` >= 2.5.0 by running `gem update --system` followed by `gem --version`
38+
* Install `Bundler 2` with `gem install bundler`
39+
* Install `Node` https://nodejs.org/en/download/
40+
* Install `Yarn` https://yarnpkg.com/en/docs/install
41+
* Run server, `make dev`
42+
* Visit http://localhost:4000/docs/
43+
44+
## 1. <a name='ChangingaDevCenterDestinationsname'></a>Changing a DevCenter Destination's name
45+
46+
Occasionally, a destination will change names. This shouldn't be too difficult to handle, but make sure you do the following:
47+
- Change the name of the file **to match destination's new slug**
48+
- Check in the Partner Portal that the name change has appropriately filled out the `previousNames` field. There should be two (or more if this has aliases/many name changes).
49+
- Add a `redirect_from` frontmatter item, with the url of the old doc. This funnels anyone arriving at the old page from a link outside the docs site to the page at the new name.
50+
- Run a `make catalog` to pick up the name change.
51+
- Run `make docs` and test that:
52+
1. The page shows up correctly at the url you specified using the new slug.
53+
2. The programmatic content appears (cmodes, settings, previous names)
54+
3. The redirect from the old page URL works.
55+
56+
## 2. <a name='AllabouttheCatalogscript'></a>All about the Catalog script
57+
58+
You run the Catalog update script by running `make catalog` from the docs repo. You, a person who is going to run the script, must first save a Segment token to an `.env` file locally, which is `gitignored` so we don’t check it in to gihub accidentally.
59+
60+
Note: Old ConfigAPI tokens are not compatible with Public API. You'll need a new one if you want to use Public API.
61+
62+
The script takes your token, inserts it into a request header, then contacts the API, downloads all the available metadata for destinations and sources. It then writes them into a series of yml files that the docs build can consume. (You can find these in `/src/_data/catalog/`)
63+
64+
Note: We don’t currently (Feb '21) do this for warehouses (storage dests) because they were originally lumped in with destinations, and didn’t change often enough to be worth writing a script for. We just have a static `warehouses.yml` file instead. With the switch to Public API from ConfigAPI, we should change this.
65+
66+
The script also “calculates” the values for the `connection-modes` table for destinations, but that’s a huge other headache.
67+
68+
It also does some slugification and destination-name normalization, since our handling of dots and dashes hasn't been consistent over time. Finally, it checks to see if there’s a folder for each destination. If it finds a new one, the script makes a folder with a “stub” markdown file for that destination, and then adds a line for it to an "incompleteDocs.txt" file. (It doesn't check to see if it's already listed, just appends to the file.)
69+
70+
### 2.1. <a name='ConnectionModesintheCatalogscript'></a>Connection Modes in the Catalog script
71+
72+
As part of the Dossiers project we worked on making the Connection Modes table more readable. Originally we were going to have per-page liquid run, but these modes don't change often so it would've added a lot of build time for very little benefit. Instead we pushed it into `catalog.js`.
73+
Once the connection modes device and cloud arrays are set, we do a bunch of calculations, and add a text summary, a number which corresponds to that summary for easier programmatic writing, and a rough category.
74+
75+
| Case | Summary | Type | Message |
76+
| ---- | -------------------------------------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
77+
| 0 | No data available | none | No connection mode information available. |
78+
| 1 | Both device, no cloud | device-only | accepts device-mode data from both Analytics.js and mobile sources. It does not accept data in cloud-mode. |
79+
| 2 | AJS (web device) only | device-only | accepts device-mode data only from Analytics.js. |
80+
| 3 | Mobile device mode only | device-only | accepts device-mode data only from a mobile source. |
81+
| 4 | Accepts from all | all | accepts cloud-mode data from all Segment source types. It can accept device-mode data from both web and mobile sources. |
82+
| 5 | All cloud types | cloud-only | accepts cloud-mode data from all Segment source types. It does not offer device-mode connections. |
83+
| 6 | Mobile and Server cloud only | cloud-only | accepts data from any Segment mobile or server source in cloud mode. It does not accept data from a web source, and does not offer device-mode connections. |
84+
| 7 | Web and mobile cloud only | cloud-only | accepts only cloud-mode data from web and mobile sources. |
85+
| 8 | Mobile cloud only | cloud-only | accepts only cloud-mode data from mobile sources. |
86+
| 9 | All cloud types, 1 device mode | mixed | accepts data in cloud-mode from all source types, and can accept data in device-mode from [Analytics.js or mobile] sources. |
87+
| 10 | Web and mobile cloud, 1 device | mixed | accepts data in cloud-mode from web and mobile sources, and can accept data in device-mode from [Analytics.js or mobile] sources. |
88+
| 11 | Mobile and server cloud, mobile device | mixed | accepts data in cloud-mode from mobile and server sources, and can accept data in device-mode from mobile sources. |
89+
90+
## 3. <a name='Developerinformation'></a>Developer information
91+
92+
93+
### 3.1. <a name='Layouts'></a>Layouts
94+
95+
`default.html` is the base container through which all the individual other layouts are built to have the right title, seo, etc. The template inheritance is described in the diagram below.
96+
97+
The `destination.html`, `source.html`, and `integration.html` templates contain the logic that runs the layouts for individual catalog pages. Storage/warehouses use the generic Integration right now because they don't need anything special. Set the layout in the Jekyll `_config.yml` file.
98+
99+
```text
100+
default.html
101+
|- integration.html
102+
|- destination.html
103+
|- source.html
104+
|- main.html
105+
|- catalog.html - used for connections catalog pages only
106+
|- home.html - for main landing page only
107+
|- page.html - used for all pages outside Connections catalogs, without an explicit override
108+
|- search.html - search results page only
109+
```
110+
111+
### 3.2. <a name='ConfigAPICatalog'></a>Config API + Catalog
112+
113+
The Segment Config API provides the data for the Source and Destination catalog pages. This happens on demand using `make catalog`, and the results are stored in the respective `_data/catalog` yml files.
114+
115+
Warehouses.yml is currently built by hand, because warehouses have traditionally been considered a form of destination, so are not separated out in the Config API.
116+
117+
#### 3.2.1. <a name='APIKey'></a>API Key
118+
The Config API needs an API key to pull in the _latest_ catalog data and currently looks for one in the environment variable `PLATFORM_API_TOKEN`. This value is stored in a special file named `.env` that the appropriate scripts reference. You can what this file looks like by looking at `.env.example`
119+
120+
If you want to interact with the Platform API, locally, first make sure you have run `make env`. This will create the appropriate `.env` file for you to work with
121+
122+
**NOTE: Never check-in `.env` or remove it from `.gitignore`.**
123+
124+
Once your local environment is configured, you then have two options to pull Platform API data: You can use the token in [`chamber`](https://github.com/segmentio/chamber) or you can create your own token.
125+
126+
127+
##### Bring your own token
128+
129+
You create your own token in the Segment App. You can use your own personal workspace, or if you have access to them, use [`segment-engineering`](https://app.segment.com/segment-engineering/settings/access-management) or [`segment_prod`](https://app.segment.com/segment_prod/settings/access-management). Go to **Settings > Access Management > Tokens**.
130+
Any type of token will work, but you might want to limit it to a read-only token. If you're working in a shared workspace, make sure you label it so folks know what it's for and don't revoke it.
131+
132+
Once you make a new token, paste the token value in the `.env` file like so:
133+
134+
```text
135+
PLATFORM_API_TOKEN=(token value here)
136+
```
137+
You can now run `make catalog`!
138+
139+
##### Chamber
140+
141+
If you installed and have access to `chamber`, run the following command:
142+
143+
```bash
144+
$ aws-okta exec prod-privileged -- chamber read segment-docs platform_api_key
145+
```
146+
147+
or for staging...
148+
149+
```bash
150+
$ aws-okta exec stage-privileged -- chamber read segment-docs platform_api_key
151+
```
152+
153+
You should get something like this as the output of the command.
154+
```bash
155+
Key Value Version LastModified User
156+
platform_api_key [REDACTED FOR DOCS] 2 08-05 10:24:55 arn:aws:sts::752180062774:assumed-role/production-write/[email protected]
157+
```
158+
159+
Edit the `.env` file (generated from `make env`) and replace the environment variable with the token above. `make catalog` should then work and you should see some output like this:
160+
161+
```bash
162+
$ make catalog
163+
"Saving catalogs from Platform API..."
164+
"Finished Destinations."
165+
"Finished Sources."
166+
"Done."
167+
```
168+
169+
170+
171+
#### 3.2.2. <a name='CatalogDataDocLinks'></a>Catalog Data + Doc Links
172+
By default, the links on the catalog page and respective sidenavs attempt to automagically set hyperlinks, for actual doc file, at the path `connections/:type/:slug`. However, given the transitory state of Docs V2, these links might 404 since the respective doc might be in a different directory.
173+
174+
#### 3.2.3. <a name='ObjectSourcesandWarehouses'></a>Object Sources and Warehouses
175+
These two catalogs are hardcoded in the `_data` directory since the Config API does not expose these resources.
176+
177+
### 3.3. <a name='Sidenav'></a>Sidenav
178+
The sidenav is managed by the files in `_data/sidenav/`. Depending on what section we are in determines the file used. We currently support up to 2 levels deep on a sidenav.
179+
180+
### 3.4. <a name='Breadcrumb'></a>Breadcrumb
181+
The current breadcrumb is currently determined based on the `page.path` and the current page's `title` front-matter attribute.
182+
183+
### 3.5. <a name='Searching'></a>Searching
184+
We're currently using Algolia, which uses `algolia.js` and `algolia.css`. The search indexing is managed on the Algolia site.
185+
186+
## 4. <a name='Testing'></a>Testing
187+
188+
### 4.1. <a name='BuildTesting'></a>Build Testing
189+
Currently the only automatic testing we perform is linting on the configuration yaml files to ensure proper the project will build.
190+
191+
TODO: define rules for markdown linting and clean up linting errors
192+
`npx remark ./src --use preset-lint-markdown-style-guide`
193+
194+
### 4.2. <a name='ManualTesting'></a>Manual Testing
195+
196+
There are also some manual testing scripts that you can run to validate the build.
197+
198+
1. Vale. Vale is a prose linter which you can run on-demand, or configure to work with Atom or other editors.
199+
200+
2. `tests/redirects/redirects_bash`: used for validating a list of paths that we have nginx redirects for
201+
202+
3. `tests/externalLinks/linkTester_bash`: used to validate that external links referenced in docs point to a validate endpoint
203+
204+
4. `tests/imageSizes/getImageSizes.js`: used to get the 10 largest images in the repo.
205+
206+
5. `npx mdspell 'src/**/*.md' -r --en-us`: used to validate spelling in docs, needs to be configured to add Segment terms.
207+
208+
6. Included is the [Hyperlink](https://www.npmjs.com/package/hyperlink) NPM module. Run `bundle install` to install that, plus the tap-spot plugin for pretty output. To check all links on the site, prior to build, run `yarn run hyperlink ./_site/index.html --canonicalroot https://segment.com/docs -i -r --skip 0.0.0.0 | yarn run tap-spot`. This module checks hyper links, images, and anchor tags to ensure that everything linked internally resolves to a location. **TODO**: Add support for external links.

0 commit comments

Comments
 (0)