Skip to content

Commit cb22686

Browse files
jlwalke2jlwalke2
authored
Dev v1.5 (#53)
* include python 3.8 * fix KeyError * fix KeyError * remove lightgbm restriction * disable checks for Py3.8 * versionadded * custom wrapper function name * wrappers for sklearn methods * drop class inheritance from object to avoid @wraps error in 2.7 * Revert "drop class inheritance from object to avoid @wraps error in 2.7" This reverts commit 0259e46 * updated target code * updated predict_proba output * moved pymas setup to package init() * fix for py27 * multiple wrapped functions * remove double counted vars * updated target * no return code/message by default * prep for release * content-type during file uploads * specify response format * cleanup * cleanup * allow rc and msg in output variables * added tests * fixed binary file content * added relationships service * added cassette * determine algorithm * improved metadata from astore * added neural net * added svm * binary gradboost model * added bayes net * cleanup * improved astore registration * added reports and report_images service * added reports and report_images service * enable testing against live server * moved dataset fixtures * explicitly load table * explicitly load table * cleanup * fixed case * cleanup for live tests * change score() to predict() * cleanup * assorted bugfixes & refreshed cassettes * added pagingiterator * require futures for py27 * updated requirements in docs * test PagingIterator * test param combinations * update doc * fixed performance task * refactoring for PagedList implementation * fixes for py27 * add iterator next() methods for py27 * py27 compatibility * debug * debug * debug * implement __getslice__ * cleanup * py27 compatibility * PagedList changes * cleanup & assorted fixes * added example data * enable passing dataframes and arrays directly to module steps * bug fix * pass pd.Series instance * added dataset readme * drop .sas file extensions * test JSON output from CLI * add --format option * fixed error formatting * improved parsing of target level * improved parsing of target level * CLI improvements * handle single-row dataframe input * recoded astore pipeline * sklearn pipeline * verify unpickle * render individual elements * activate deepsource * release file handles * add tests * cleanup * cleanup * cleanup * cleanup * code qual updates * remove list comprehension * code quality * code quality * code quality * fix len() comparisons * code qual * code qual * perf improvements * cleanup * cleanup * refresh cassettes * ignore private files * ds2 package constructor * bug fix * skip mas publish * add cassette * code qual improvements * code qual improvements * code qual improvements * fix for py27 * check swat build * fix py27 error * check pandas compatibility * code qual improvements * code qual improvements * code qual improvements * consistent project creation in register_model * fix KeyErrors * fixed delete_destination * cleanup * cleanup * include basic REST functions * add iris dataset * use local dataset * fix astore model inputs * cleanup * updated pymas wrapper * convert numpy values to scalar * do not wrap predict_proba method by default. * handle string and numeric outputs * handle predict_proba methods. deprecated return_msg and return_code. * updated targets * fixed tests * fixed formatting * fixed formatting * code quality * fixed astore example * fixed full lifecycle example * test sklearn classification model * any valid classification * code quality * code quality * fix for py 2.7 * added SGF examples * updated for py27 * initial draft * record cassettes for scenarios * initial draft * trim spaces * suppress pickle warnings * renamed files * updated list_folders example * HATEOAS section * skipcq * cleanup * enabled additional tests * enable tests on non-master branches * prep for release
1 parent 36a523d commit cb22686

File tree

207 files changed

+155721
-17320
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

207 files changed

+155721
-17320
lines changed

‎.deepsource.toml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,14 @@
11
version = 1
22

3+
test_patterns = [
4+
"tests/**"
5+
]
6+
7+
exclude_patterns = [
8+
"tests/**"
9+
]
10+
11+
312
[[analyzers]]
413
name = "python"
514
enabled = true

‎.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
## Assorted files that shouldn't be uploaded
2+
.private/*
3+
_private/*
4+
15
## Editor temporary/working/backup files
26
.#*
37
*\#*\#

‎.travis.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,9 @@ matrix:
3434
env: TOXENV=py36-tests-codecov TOX_SKIP_MISSING_INTERPRETERS=False
3535
- python: 3.7
3636
env: TOXENV=py37-tests-codecov TOX_SKIP_MISSING_INTERPRETERS=False
37+
# TODO: Enable when numpy, Pandas, and other dependencies are released on 3.8
38+
# - python: 3.8
39+
# env: TOXENV=py38-tests-codecov TOX_SKIP_MISSING_INTERPRETERS=False
3740

3841

3942
- stage: flake8

‎CHANGELOG.md‎

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,27 @@
11
Unreleased
22
----------
3-
-
3+
-
4+
5+
v1.5 (2020-2-23)
6+
----------------
7+
**Improvements**
8+
- Registered Python models will now include both `predict` and `predict_proba` methods.
9+
- Added a new Relationships service for managing links between objects.
10+
- Added a new Reports service for retrieving SAS Visual Analytics reports.
11+
- Added a new Report_Images service for rendering content from reports.
12+
- Additional metadata fields are set when registering an ASTORE model.
13+
- Collections of items should now return an instance of `PagedList` for lazy loading of results.
14+
- Module steps can now be called using `module.step(df)` where `df` is the row of a DataFrame or Numpy array.
15+
- `register_model` sets additional project properties when registering an ASTORE model.
16+
17+
**Changes**
18+
- Replaced the `raw` parameter of the `request` methods with a `format` parameter, allowing more control over the
19+
returned value.
20+
- The `get_file_content` method of the Files service now returns the actual content instead of the file metadata.
21+
- JSON output when using `sasctl` from the command line is now formatted correctly.
22+
23+
**Bugfixes**
24+
- `model_publish.delete_destination` now works correctly.
425

526
v1.4.6 (2020-1-24)
627
------------------

‎README.md‎

Lines changed: 12 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ sasctl requires the following Python packages be installed.
3939
If not already present, these packages will be downloaded and installed automatically.
4040
- requests
4141
- six
42+
- futures (Python 2.7 only)
4243

4344
The following additional packages are recommended for full functionality:
4445
- swat
@@ -88,29 +89,24 @@ which can handle all necessary communication with the SAS Viya server:
8889
A slightly more low-level way to interact with the environment is to use
8990
the service methods directly:
9091
```
91-
>>> from pprint import pprint
9292
>>> from sasctl import Session
9393
>>> from sasctl.services import folders
9494
9595
>>> with Session(host, username, password):
96-
... folders = folders.list_folders()
97-
... pprint(folders)
98-
99-
{'links': [{'href': '/folders/folders',
100-
'method': 'GET',
101-
'rel': 'folders',
102-
'type': 'application/vnd.sas.collection',
103-
'uri': '/folders/folders'},
104-
{'href': '/folders/folders',
105-
'method': 'POST',
106-
'rel': 'createFolder',
96+
... for f in folders.list_folders():
97+
... print(f)
98+
99+
Public
100+
Projects
101+
ESP Projects
102+
Risk Environments
107103
108104
... # truncated for clarity
109105
110-
'rel': 'createSubfolder',
111-
'type': 'application/vnd.sas.content.folder',
112-
'uri': '/folders/folders?parentFolderUri=/folders/folders/{parentId}'}],
113-
'version': 1}
106+
My Folder
107+
My History
108+
My Favorites
109+
SAS Environment Manager
114110
```
115111

116112

‎doc/api/services/relationships.rst‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
sasctl.services.relationships
2+
=============================
3+
4+
.. automodule:: sasctl._services.relationships
5+
:members:
6+
:undoc-members:
7+
:show-inheritance:

‎doc/api/services/report_images.rst‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
sasctl.services.report_images
2+
=============================
3+
4+
.. automodule:: sasctl._services.report_images
5+
:members:
6+
:undoc-members:

‎doc/api/services/reports.rst‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
sasctl.services.reports
2+
=======================
3+
4+
.. automodule:: sasctl._services.reports
5+
:members:
6+
:undoc-members:

‎doc/index.rst‎

Lines changed: 81 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ If not already present, these packages will be downloaded and install automatica
2929

3030
- requests
3131
- six
32+
- futures (Python 2.7 only)
3233

3334
The following additional packages are recommended for full functionality:
3435

@@ -73,40 +74,39 @@ handle all necessary communication with the SAS Viya server::
7374

7475
>>> with Session('example.com', authinfo=<authinfo file>):
7576
... model = lm.LogisticRegression()
76-
... register_model('Sklearn Model', model, 'My Project')
77+
... register_model(model, 'Sklearn Model', 'My Project')
7778

7879

7980
A slightly more low-level way to interact with the environment is to use the service
8081
methods directly::
8182

82-
>>> from pprint import pprint
8383
>>> from sasctl import Session
8484
>>> from sasctl.services import folders
8585

8686
>>> with Session(host, username, password):
87-
... folders = folders.list_folders()
88-
... pprint(folders)
87+
... for f in folders.list_folders():
88+
... print(f)
8989

90-
{'links': [{'href': '/folders/folders',
91-
'method': 'GET',
92-
'rel': 'folders',
90+
Public
91+
Projects
92+
ESP Projects
93+
Risk Environments
9394

9495
... # truncated for clarity
9596

96-
'rel': 'createSubfolder',
97-
'type': 'application/vnd.sas.content.folder',
98-
'uri': '/folders/folders?parentFolderUri=/folders/folders/{parentId}'}],
99-
'version': 1}
100-
97+
My Folder
98+
My History
99+
My Favorites
100+
SAS Environment Manager
101101

102102
The most basic way to interact with the server is simply to call REST functions
103103
directly, though in general, this is not recommended.::
104104

105105
>>> from pprint import pprint
106-
>>> from sasctl import Session
106+
>>> from sasctl import get, Session
107107

108-
>>> with Session(host, username, password) as s:
109-
... folders = s.get('/folders')
108+
>>> with Session(host, username, password):
109+
... folders = get('/folders')
110110
... pprint(folders)
111111

112112
{'links': [{'href': '/folders/folders',
@@ -169,21 +169,62 @@ This also works on individual commands::
169169
Common Uses
170170
++++++++++++
171171

172-
Registering a model built with SWAT.
172+
Register a SAS model
173+
~~~~~~~~~~~~~~~~~~~~~~~
174+
175+
::
176+
177+
import swat
178+
from sasctl import register_model, Session
179+
180+
with swat.CAS('hostname', 5570, 'username', 'password') as cas:
181+
astore = cas.CASTable('model_astore_table')
182+
183+
Session('hostname', 'username', 'password'):
184+
185+
model = register_model(astore, 'Model Name', 'Project Name')
173186

174-
.. literalinclude:: ../examples/astore_model.py
175-
:caption: examples/astore_model.py
176-
:lines: 7-
177187

178-
Registering a model built with sci-kit learn.
188+
Register a scikit-learn model
189+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
179190

180-
.. literalinclude:: ../examples/sklearn_model.py
181-
:caption: examples/sklearn_model.py
182-
:lines: 7-
191+
::
192+
193+
from sklearn.linear_model import LogisticRegression
194+
from sasctl import register_model, Session
195+
196+
model = LogisticRegression()
197+
model.fit(X, y)
198+
199+
# Establish a session with Viya
200+
with Session('hostname', 'username', 'password'):
201+
register_model(model, 'Model Name', 'Project Name', input=X)
202+
203+
204+
Publish a model
205+
~~~~~~~~~~~~~~~
206+
207+
::
208+
209+
from sasctl import publish_model, Session
210+
from sasctl.services import model_repository as mr
211+
212+
model = mr.get_model('Model Name')
213+
publish_model(model, 'Destination Name')
214+
215+
216+
Execute a model in MAS
217+
~~~~~~~~~~~~~~~~~~~~~~
218+
219+
::
220+
221+
from sasctl import Session
222+
from sasctl.services import microanalytic_score as mas
223+
224+
module = mas.get_module('Module Name')
225+
module.predict(model_inputs)
183226

184227

185-
- publish a model
186-
- score a model
187228

188229
See the :file:`examples/` directory in the repository for more complete examples.
189230

@@ -274,8 +315,22 @@ by updating this logger, or the ``sasctl.core.session`` logger can be configured
274315
HATEOAS
275316
+++++++
276317

277-
Coming soon.
318+
Many of the SAS microservices follow the `HATEOAS`_ paradigm and the standard is for services to return a links
319+
collection containing valid operations. Most **sasctl** operations return one or more instances of :class:`~sasctl.core.RestObj`. Any
320+
links related to that object are accessible via the ['links'] key. Each link is represented as a dictionary containing metadata::
321+
322+
{'method': 'POST', 'rel':
323+
'createFolder',
324+
'href': '/folders/folders',
325+
'uri': '/folders/folders',
326+
'type': 'application/vnd.sas.content.folder'}
327+
328+
However, instead of having to parse this collection to find a link, **sasctl** includes some functions to make this easy: :func:`~sasctl.core.get_link` and :func:`~sasctl.core.request_link`.
329+
330+
Given an object and a link name (`rel`) :func:`~sasctl.core.get_link` will return the metadata for that link. Similarly,
331+
:func:`~sasctl.core.request_link` will make the request to the link and return the response object.
278332

333+
.. _`HATEOAS`: https://restfulapi.net/hateoas/
279334

280335
API Reference
281336
-------------

‎examples/astore_model.py‎

Lines changed: 0 additions & 31 deletions
This file was deleted.

0 commit comments

Comments
 (0)