Skip to content

Commit 02706f0

Browse files
committed
Updated readme lead and added metadata section.
1 parent c43e4b8 commit 02706f0

File tree

1 file changed

+15
-9
lines changed

1 file changed

+15
-9
lines changed

README.md

Lines changed: 15 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -11,18 +11,16 @@ Porter <img src="https://github.com/ScriptFUSION/Porter/wiki/images/porter%20222
1111
[![][Porter transformers icon]][Porter transformers]
1212
[![][Porter connectors icon]][Porter connectors]
1313

14+
Porter [reliably](#durability) and efficiently imports data into applications. Her interfaces pass around **iterators**, herein called [*record collections*](#record-collections), allowing us to process large data sets sequentially without loading every record into memory at once. She is commonly used to integrate third party Web APIs, but strives to import any data from anywhere.
1415

15-
Porter's principle goal is to [reliably](#durability) and efficiently import data into applications. Porter's interfaces pass **arrays**, called *records* via **iterators**, called [*record collections*](#record-collections). Since records may embed any data type and record collections may iterate over any number of records, Porter can theoretically stream any data format of any size in a memory efficient manner.
16+
Data does not always arrive in the format we want, so Porter can apply a series of [transformations](#transformers) during import. Transformations can perform simple [filtering](#filtering) to powerful data manipulations such as [joining linked data sets][Sub-imports] together.
1617

17-
Data does not always arrive in the format we want, so Porter can apply a series of [transformations](#transformers) during import. Transformations can perform powerful data manipulations such as [joining linked data sets][Sub-imports] together. She is commonly used to integrate third party APIs, but strives to import all kinds of data from anywhere.
18-
19-
The [Provider organization][Provider] hosts ready-to-use Porter providers to help quickly gain access to popular third-party APIs and data services. As examples, the [Stripe provider][Stripe provider] allows an application to make online payments, whereas the [European Central Bank provider][ECB provider] imports the latest currency exchange rates. Anyone writing new providers is encouraged to contribute them to the organization to share with other Porter users.
18+
The [Provider organization][Provider] hosts ready-to-use Porter providers to help quickly gain access to popular third-party APIs and data services. For example, the [Stripe provider][Stripe provider] allows an application to make online payments, whereas the [European Central Bank provider][ECB provider] imports the latest currency exchange rates. You're encouraged to contribute new providers to the organization to share with other Porter users.
2019

2120
Contents
2221
--------
2322

2423
1. [Audience](#audience)
25-
1. [Benefits](#benefits)
2624
1. [Quick start](#quick-start)
2725
1. [Usage](#usage)
2826
1. [Porter's API](#porters-api)
@@ -45,10 +43,9 @@ Contents
4543
Audience
4644
--------
4745

48-
Porter is useful for anyone wanting a [simple API](#porters-api) to import data into PHP applications. Data typically comes from third-party APIs, but it could come from any source, including web scraping or even first-party APIs, using Porter to consume our own data services. Porter is a uniform way to abstract importing data, with benefits.
46+
Porter is a useful abstraction for anyone wanting a [simple](#porters-api) and [robust](#durability) API to import data into PHP applications. Data typically comes from third-party APIs, but could come from any source, including web scraping, first-party APIs (using Porter to consume our own data services) or even the local file system. Porter is a uniform way to abstract importing data with the following benefits.
4947

50-
Benefits
51-
--------
48+
### Benefits
5249

5350
* Provides a [framework](#architecture) of structured data import concepts: [providers](#providers) provide data via [resources](#resources) fetched from [connectors](#connectors).
5451
* Defines efficient in-memory data processing interfaces to handle large data sets.
@@ -126,7 +123,7 @@ Options may be configured by some of the methods listed below.
126123
Record collections
127124
------------------
128125

129-
Record collections are a type of `Iterator`, guaranteeing imported data is enumerable using `foreach`. The result of a successful `Porter::import` call is either an instance of `PorterRecords` or `CountablePorterRecords`, depending on whether the number of records is known. That's all you need to know! The following details are just for debugging and nerds.
126+
Record collections are a type of `Iterator`, guaranteeing imported data is enumerable using `foreach`. Each *record* of the collection is the familiar and flexible `array` type, allowing us to represent any flat or structured data hierarchy (like CSV or JSON) as an array.
130127

131128
### Details
132129

@@ -136,6 +133,12 @@ Record collections are composed by Porter using the decorator pattern. If provid
136133

137134
The stack of record collection types informs us of the transformations a collection has undergone and each type holds a pointer to relevant objects that participated in the transformation. For example, `PorterRecords` holds a reference to the `ImportSpecification` that was used to create it and can be accessed using `PorterRecords::getSpecification`.
138135

136+
### Metadata
137+
138+
Since record collections are just objects, it is possible to define derived types that implement custom fields to expose additional *metadata* in addition to the iterated data. Collections are very good at representing a repeating data series but some APIs send additional non-repeating data which we can expose as metadata. However, if the data is not repeating at all, it should be treated as a single record rather than metadata.
139+
140+
The result of a successful `Porter::import` call is always an instance of `PorterRecords` or `CountablePorterRecords`, depending on whether the number of records is known. If we need to access methods of the original collection, returned by the provider, we can call `findFirstCollection()` on the collection. For an example, see [CurrencyRecords][CurrencyRecords] of the [European Central Bank Provider][ECB] and its associated [test case][ECB test].
141+
139142
Transformers
140143
------------
141144

@@ -481,3 +484,6 @@ Porter is published under the open source GNU Lesser General Public License v3.0
481484
[Porter transformers icon]: https://avatars2.githubusercontent.com/u/24607042?v=3&s=35 "Porter transformers"
482485
[Porter connectors icon]: https://avatars3.githubusercontent.com/u/25672142?v=3&s=35 "Porter connectors"
483486
[Class diagram]: https://github.com/ScriptFUSION/Porter/wiki/images/diagrams/Porter%20UML%20class%20diagram%203.0.png
487+
[ECB]: https://github.com/Provider/European-Central-Bank
488+
[CurrencyRecords]: https://github.com/Provider/European-Central-Bank/blob/master/src/Records/CurrencyRecords.php
489+
[ECB test]: https://github.com/Provider/European-Central-Bank/blob/master/test/ForeignExchangeReferenceRatesTest.php

0 commit comments

Comments
 (0)