Skip to content

Commit a603258

Browse files
fvankriekenfvankrieken
committed
saf_s
1 parent 43e8ab4 commit a603258

File tree

10 files changed

+148
-3
lines changed

10 files changed

+148
-3
lines changed

‎products/cscl/design_doc.md‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -349,6 +349,31 @@ All output categories (collection of SAF types) have their own fields and format
349349
|SAFI7|saftype|SAF Record Type Code|1|86|86|RJSF|false|
350350
||filler_safi7|Filler|12|87|98|RJSF|false|
351351

352+
`SAFS` formatting
353+
|fic|field_name|field_label|field_length|start_index|end_index|justify_and_fill|blank_if_none|
354+
|---|----------|-----------|------------|-----------|---------|----------------|-------------|
355+
|SAFS1|place_name|Street Name|32|1|32|LJSF|false|
356+
|SAFS2|boroughcode|Borough Code|1|33|33|RJSF|false|
357+
|SAFS3|face_code|Face Code|4|34|37|RJZF|false|
358+
|SAFS4|segment_seqnum|Sequence Number|5|38|42|RJZF|false|
359+
|SAFS5|sos_indicator|SOS Indicator|1|43|43|RJSF|false|
360+
|SAFS6|b5sc|B5SC|6|44|49|RJSF|false|
361+
|SAFS7|hn|HN Basic|7|50|56|RJSF|false|
362+
||filler_safs7|Filler|1|57|57|RJSF|true|
363+
|SAFS8|hn_suffix|House Number Suffix|8|58|65|LJSF|true|
364+
||filler_safs8|Filler|9|66|74|RJSF|false|
365+
|SAF85|high_alpha_hn_suffix|High Alpha House number suffix|1|75|75|RJSF|false|
366+
||filler_saf85|Filler|10|76|85|RJSF|false|
367+
|SAFS9|saftype|SAF Record Type Code|1|86|86|RJSF|false|
368+
||filler_safs9|Filler|2|87|88|RJSF|false|
369+
|SAFS10|lgc1|LGC1|2|89|90|RJZF|false|
370+
|SAFS11|lgc2|LGC2|2|91|92|RJZF|true|
371+
|SAFS12|lgc3|LGC3|2|93|94|RJZF|true|
372+
|SAFS13|lgc4|LGC4|2|95|96|RJZF|true|
373+
|SAFS14|boe_lgc_pointer|Pointer to BOE Preferred LGC|1|97|97|RJSF|false|
374+
|SAFS15|segment_type|Segment Type Code|1|98|98|RJSF|false|
375+
|SAFS16|segmentid|Segment ID|7|99|105|RJZF|false|
376+
352377
`SAFV` formatting
353378
|fic|field_name|field_label|field_length|start_index|end_index|justify_and_fill|blank_if_none|
354379
|---|----------|-----------|------------|-----------|---------|----------------|-------------|
@@ -647,5 +672,35 @@ Notes on house numbers: in processing in `int__address_points`, all house number
647672
- for `hyphen_type` = 'N', 'Q', 'U', AddressPoint cleaned/formatted `house_number_suffix` field
648673
- for `hyphen_type` = 'R', 'X', AddressPoint field `house_number_range_suffix`
649674

675+
#### S Records
676+
S records are suffixed house number(s) occurring at an intersection.
677+
678+
They are generated from all address point records with `special_condition` = 'S'.
679+
680+
Starting from `int__saf_segments`, S records are joined to
681+
- address points (`int__address_points`) to lookup primary tabular data for the saf record
682+
- `stg__facecode_and_featurename` is joined by actual b7sc. Unlike type V records, vanity b7sc is not used at all
683+
- address point lgc table (`int__lgc_address_point`), a lookup table of address points to lgcs based on `addresspointid`
684+
685+
| FIC | Field | CSCL Source of Data |
686+
| - | - | - |
687+
| SAFS1 | Street Name | Obtained from FeatureName table, in sort format |
688+
| SAFS2 | Borough Code | AddressPoint field `boroughcode` |
689+
| SAFS3 | Face Code | Obtained from FeatureName table |
690+
| SAFS4 | Sequence Number | Segment `segment_seqnum` |
691+
| SAFS5 | SOS Indicator | AddressPoint field `sosindicator`, mapped `1` -> `L`, `2` -> `R` |
692+
| SAFS6 | B5SC | First 6 bytes of AddressPoint field `b7sc_actual` |
693+
| SAFS7 | HN Basic | 7 | 50 | 56 | RJSF | AddressPoint field `house_number` |
694+
| SAFS8 | House Number Suffix | 8 | 58 | 65 | LJSF, blanks if null | AddressPoint field `house_number_suffix` |
695+
| SAF85 | High Alpha House number suffix | 1 | 75 | 75 | AddressPoint field `house_number_RANGE_suffix` |
696+
| SAFS9 | SAF Record Type Code | 1 | 86 | 86 | AddressPoint field `special_condition` |
697+
| SAFS10 | LGC1 | Segment `lgc1` |
698+
| SAFS11 | LGC2 | Segment `lgc2` |
699+
| SAFS12 | LGC3 | Segment `lgc3` |
700+
| SAFS13 | LGC4 | Segment `lgc4` |
701+
| SAFS14 | Pointer to BOE Preferred LGC | Segment `boe_lgc_pointer` |
702+
| SAFS15 | Segment Type Code | Segment `segment_type` |
703+
| SAFS16 | Segment ID | Segment `segment_id` |
704+
650705
# Infrastructure
651706
[stub]

‎products/cscl/models/intermediate/saf/int__address_points.sql‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ street_names AS (
1414
)
1515
SELECT
1616
address_points.addresspointid,
17+
address_points.segmentid,
1718
address_points.b7sc_vanity,
1819
address_points.b7sc_actual,
1920
address_points.sosindicator,
@@ -30,9 +31,8 @@ SELECT
3031
ELSE address_points.house_number
3132
END AS house_number,
3233
CASE
33-
WHEN street_names.snd_feature_type IN ('E', 'F') THEN NULL
34-
WHEN address_points.hyphen_type = 'R'
35-
THEN TRIM(SPLIT_PART(address_points.house_number, '-', 2))
34+
WHEN street_names.snd_feature_type IN ('E', 'F')
35+
THEN TRIM(SPLIT_PART(address_points.house_number_suffix, '-', 1))
3636
ELSE address_points.house_number_suffix
3737
END AS house_number_suffix,
3838
address_points.house_number AS plain_house_number,

‎products/cscl/models/intermediate/saf/int__saf_s.sql‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
WITH saf AS (
2+
SELECT * FROM {{ ref('int__saf_segments') }}
3+
),
4+
address_points AS (
5+
SELECT * FROM {{ ref('int__address_points') }}
6+
),
7+
feature_names AS (
8+
SELECT * FROM {{ ref('stg__facecode_and_featurename') }}
9+
),
10+
lgc AS (
11+
SELECT * FROM {{ ref('int__streetcode_and_facecode') }}
12+
)
13+
SELECT
14+
feature_names.saf_place_name AS place_name,
15+
saf.boroughcode,
16+
COALESCE(feature_names.face_code, SUBSTRING(saf.segment_lionkey, 2, 4)) AS face_code,
17+
SUBSTRING(saf.segment_lionkey, 6, 5) AS segment_seqnum, -- TODO is this fine for generic/roadbed?
18+
CASE
19+
WHEN address_points.sosindicator = '1' THEN 'L'
20+
WHEN address_points.sosindicator = '2' THEN 'R'
21+
END AS sos_indicator,
22+
LEFT(address_points.b7sc_actual, 6) AS b5sc,
23+
address_points.house_number AS hn,
24+
address_points.house_number_suffix AS hn_suffix,
25+
address_points.house_number_range_suffix AS high_alpha_hn_suffix,
26+
saf.saftype,
27+
lgc.lgc1,
28+
lgc.lgc2,
29+
lgc.lgc3,
30+
lgc.lgc4,
31+
lgc.boe_lgc_pointer,
32+
saf.segment_type,
33+
saf.segmentid,
34+
saf.generic,
35+
saf.roadbed,
36+
address_points.addresspointid,
37+
feature_names.b7sc IS NOT NULL AS _joined_to_feature_name, -- for error report
38+
address_points.snd_feature_type
39+
FROM saf
40+
LEFT JOIN address_points ON saf.saf_globalid = address_points.globalid
41+
LEFT JOIN feature_names -- TODO error when not joined
42+
ON address_points.b7sc_actual = feature_names.b7sc AND feature_names.feature_type = 'street'
43+
LEFT JOIN lgc ON saf.segmentid = lgc.segmentid
44+
WHERE saf.saftype = 'S'

‎products/cscl/models/product/saf/_saf.yml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,3 +78,9 @@ models:
7878

7979
- name: saf_ov_roadbed
8080
columns: *dat_column_119
81+
82+
- name: saf_s_generic
83+
columns: *dat_column_105
84+
85+
- name: saf_s_roadbed
86+
columns: *dat_column_105

‎products/cscl/models/product/saf/exports/saf_s_generic.sql‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{{ select_rows_as_text(model='saf_s_generic_by_field') }}

‎products/cscl/models/product/saf/exports/saf_s_roadbed.sql‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{{ select_rows_as_text(model='saf_s_roadbed_by_field') }}

‎products/cscl/models/product/saf/saf_s_generic_by_field.sql‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
SELECT
2+
{{ apply_text_formatting_from_seed('text_formatting__saf_s') }}
3+
FROM {{ ref("int__saf_s" ) }}
4+
WHERE generic

‎products/cscl/models/product/saf/saf_s_roadbed_by_field.sql‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
SELECT
2+
{{ apply_text_formatting_from_seed('text_formatting__saf_s') }}
3+
FROM {{ ref("int__saf_s" ) }}
4+
WHERE roadbed

‎products/cscl/recipe.yml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,14 @@ exports:
107107
filename: RoadbedOV.txt
108108
format: dat
109109
custom: { formatting: saf_v }
110+
- name: saf_s_generic
111+
filename: GenericS.txt
112+
format: dat
113+
custom: { formatting: saf_s }
114+
- name: saf_s_roadbed
115+
filename: RoadbedS.txt
116+
format: dat
117+
custom: { formatting: saf_s }
110118

111119
# SEDAT
112120
- name: sedat

‎products/cscl/seeds/text_formatting/text_formatting__saf_s.csv‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
fic,field_name,field_label,field_length,start_index,end_index,justify_and_fill,blank_if_none
2+
SAFS1,place_name,Street Name,32,1,32,LJSF,FALSE
3+
SAFS2,boroughcode,Borough Code,1,33,33,RJSF,FALSE
4+
SAFS3,face_code,Face Code,4,34,37,RJZF,FALSE
5+
SAFS4,segment_seqnum,Sequence Number,5,38,42,RJZF,FALSE
6+
SAFS5,sos_indicator,SOS Indicator,1,43,43,RJSF,FALSE
7+
SAFS6,b5sc,B5SC,6,44,49,RJSF,FALSE
8+
SAFS7,hn,HN Basic,7,50,56,RJSF,FALSE
9+
,filler_safs7,Filler,1,57,57,RJSF,TRUE
10+
SAFS8,hn_suffix,House Number Suffix,8,58,65,LJSF,TRUE
11+
,filler_safs8,Filler,9,66,74,RJSF,FALSE
12+
SAF85,high_alpha_hn_suffix,High Alpha House number suffix,1,75,75,RJSF,FALSE
13+
,filler_saf85,Filler,10,76,85,RJSF,FALSE
14+
SAFS9,saftype,SAF Record Type Code,1,86,86,RJSF,FALSE
15+
,filler_safs9,Filler,2,87,88,RJSF,FALSE
16+
SAFS10,lgc1,LGC1,2,89,90,RJZF,FALSE
17+
SAFS11,lgc2,LGC2,2,91,92,RJZF,TRUE
18+
SAFS12,lgc3,LGC3,2,93,94,RJZF,TRUE
19+
SAFS13,lgc4,LGC4,2,95,96,RJZF,TRUE
20+
SAFS14,boe_lgc_pointer,Pointer to BOE Preferred LGC,1,97,97,RJSF,FALSE
21+
SAFS15,segment_type,Segment Type Code,1,98,98,RJSF,FALSE
22+
SAFS16,segmentid,Segment ID,7,99,105,RJZF,FALSE

0 commit comments

Comments
 (0)