ibmstreams / pypi.streamsx.standard

Python packages for IBM Streams Standard Toolkit

Exposing SPL standard toolkit as classes and installable package for the Python Application API for IBM Streams.

Latest documentation

• http://streamsxstandard.readthedocs.io

Package is organized using standard packaging to upload to PyPi.

• http://pypi.org/project/streamsx.standard/

pip install streamsx.standard

The latest code be installed as the streamsx.standard package directly from github using:

pip install git+https://github.com/IBMStreams/pypi.streamsx.standard.git

Test

Run the full test with:

ant test

pypi.streamsx.standard's Issues

rename repository to pypi.streamsx.standard

aligned with other pypi streamsx packages

package name and namespace must not be changed (repo name only)

Provide build script for doc and test

add targets for

• clean generated files
• build doc
• run tests

Doc page: Edit on Github link broken

Takes me to a 404 page.

replace usage of sequence function with new class interface

adapt affected test suites

FileSink: error with tuples_per_file

streamsx.standard.tests.test_files/test_filename_from_stream.spl:35:7: CDISP9150E ERROR: The type of the tuplesPerFile parameter of the FileSink operator is int32. The expected type is: uint32/uint64.

How is `streamsx.standard.files.csv_reader` meant to be used?

The file parameter is hardcoded, but for that to work the path to the file on the Streams host must be known at job submission time.

This code:

import streamsx.ec
from  streamsx.standard.files import csv_reader

from streamsx.topology.topology import Topology
import streamsx.topology.context

csv_file_name = "next_bus_avg_max_speed.csv"    

csvtopo = Topology()

csv_schema = 'tuple<int64 last_seen, float32 maxSpeed, rstring id, float32 avgSpeed>'
file = "etc/" + csv_file_name
readings = csv_reader(csvtopo, file="/Users/natashad/Downloads/next_bus_avg_max_speed.csv", schema=csv_schema, header=True, name="ReadCSV")
readings.print()

Throws a FileNotFound error.

Check parameters and test coverage

     [echo] Name                              Stmts   Miss  Cover
     [echo] -----------------------------------------------------
     [echo] streamsx/standard/__init__.py        42      0   100%
     [echo] streamsx/standard/_version.py         1      0   100%
     [echo] streamsx/standard/files.py          441     37    92%
     [echo] streamsx/standard/relational.py     118     28    76%
     [echo] streamsx/standard/utility.py        166     11    93%
     [echo] -----------------------------------------------------
     [echo] TOTAL                               768     76    90%
     [echo] ----------------------------------------------------------------------
     [echo] Ran 28 tests in 687.794s
     [echo] 
     [echo] OK

convert csv_reader/writer to composite classes

Support parameter compression in CSVReader

invoke FileSource in with parameter compression

Doc issue: incomplete note in streamsx.standard.utility.union

Documentation of Note block in streamsx.standard.utility.union is incomplete
This method differs from Stream.union in that

"make html" raises the following warning:

streamsx.standard/streamsx/standard/utility.py:docstring of streamsx.standard.utility.union:15: WARNING: Explicit markup ends without a blank line; unexpected unindent.

change default branch to develop

do not use master in branch name anymore

Update link to streamsx package in index.rst

Update link to streamsx package in index.rst:
from:
http://ibmstreams.github.io/streamsx.topology/doc/releases/latest/pythondoc/index.html
to:
https://streamsxtopology.readthedocs.io

Provide composite class for FileSink

uses streamsx feature:
IBMStreams/streamsx.topology#2322