Coder Social home page Coder Social logo

friends-of-freeswitch / switchio Goto Github PK

View Code? Open in Web Editor NEW
191.0 14.0 28.0 3.85 MB

asyncio powered FreeSWITCH cluster control

Home Page: http://switchio.rtfd.io

License: Mozilla Public License 2.0

Python 100.00%
freeswitch esl asyncio coroutines telephony stress-tester voip aio dialer auto-dialer

switchio's Introduction

switchio

asyncio powered FreeSWITCH cluster control using pure Python 3.6+

pypi github_actions versions license docs

switchio (pronounced Switch Ee OoH) is the next evolution of switchy (think Bulbasaur -> Ivysaur) which leverages modern Python's new native coroutine syntax and, for now, asyncio.

API-wise the project intends to be the flask for VoIP but with a focus on performance and scalability more like sanic.

Use the power of async and await!

Build a routing system using Python's new coroutine syntax:

from switchio.apps.routers import Router

router = Router(
    guards={
        'Call-Direction': 'inbound',
        'variable_sofia_profile': 'external'},
    subscribe=('PLAYBACK_START', 'PLAYBACK_STOP'),
)

@router.route('(.*)')
async def welcome(sess, match, router):
    """Say hello to inbound calls.
    """
    await sess.answer()  # resumes once call has been fully answered
    sess.log.info("Answered call to {}".format(match.groups(0)))

    sess.playback(  # non-blocking
        'en/us/callie/ivr/8000/ivr-founder_of_freesource.wav')
    await sess.recv("PLAYBACK_START")
    sess.log.info("Playing welcome message")

    await sess.recv("PLAYBACK_STOP")
    await sess.hangup()  # resumes once call has been fully hungup

Run this app (assuming it's in dialplan.py) from the shell:

$ switchio serve fs-host1 fs-host2 fs-host3 --app ./dialplan.py:router

You can also run it from your own script:

if __name__ == '__main__':
    from switchio import Service
    service = Service(['fs-host1', 'fs-host2', 'fs-host3'])
    service.apps.load_app(router, app_id='default')
    service.run()

Spin up an auto-dialer

Run thousands of call flows to stress test your service system using the built-in auto-dialer:

$ switchio dial fs-tester1 fs-tester2 --profile external --proxy myproxy.com --rate 100 --limit 3000

Install

pip install switchio

Docs

Oh we've got them docs!

How do I deploy my FreeSWITCH cluster?

See the docs for the deats!

What's included?

  • A slew of built-in apps
  • A full blown auto-dialer originally built for stress testing VoIP service systems
  • Super detailed ESL event logging

How can I contribute?

Have an idea for a general purpose switchio app or helper? Make a PR here on GitHub!

Also, if you like switchio let us know on Riot!

Wait, how is switchio different from other ESL clients?

switchio differentiates itself by supporting FreeSWITCH process cluster control as well as focusing on leveraging the most modern Python language features. switchio takes pride in being a batteries included framework that tries to make all the tricky things about FreeSWITCH a cinch.

What if I'm stuck on Python 2?

Check out these other great projects:

Performance monitoring

If you'd like to record performance measurements using the CDR app, some optional numerical packages can be used:

Feature Dependency Installation
Metrics Capture pandas pip install switchio[metrics]
Graphing matplotlib pip install switchio[graphing]
HDF5 pytables [1] pip install switchio[hdf5]
[1]pytables support is a bit shaky and not recommended unless you intend to locally process massive data sets worth of CDRs. The default CSV backend is usually sufficient on a modern file system.

License

All files that are part of this project are covered by the following license, except where explicitly noted.

This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of the MPL was not distributed with this file, You can obtain one at http://mozilla.org/MPL/2.0/.

switchio's People

Contributors

false-vacuum avatar goodboy avatar kick1911 avatar moises-silva avatar ncorbic-sangoma avatar steveayre avatar vodik avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

switchio's Issues

Show session uuid in Session.__repr__

Issue by tgoodlet
Friday Jul 08, 2016 at 16:19 GMT
Originally opened as sangoma/switchy#42


This way when mucking around with the session api (usually in a debugger/console without having to check Session.uuid) you can move back and forth between fs_cli and know which uuid to use when making manual api calls.

broken cli

Running switchio serve <host> <app> currently fails due to an API mismatch.
We need a test and fix.

Use caplog from pytest 3.3

Version 3.3 of pytest now supports capturing and verifying logging messages.
That's definitely useful for a bunch of tests in the suite and those tests need to be noted and adjusted to leverage this feature stat!

Docstring cleanup

A bunch of the doc strings are causing sphinx warnings like:

WARNING: repos/switchio/switchio/api.py:docstring of switchio.api.Client.set_orig_cmd:8: (SEVERE/4) Unexpected section title.

Returns
-------

Coordinate with #10 to fix these.

Don't use per connection loggers?

The logging cookbook has a good snippet on using the extra keyword to add contextual info to output. It also suggests not using Logger instances per connection like we do.

It means changing a lot of log message calls for maybe minimal benefit unless we use a LogAdapter. I'm split on what to do since generally there's only one connection / event loop per FS process in switchio and maybe it's not worth the effort?

Document the coroutine API

Need to showcase in the readme and docs!
As part of this It's likely many of the apps will be converted.

Add sendmsg support

Since moving to FreeSWITCH 1.6 I've had to disable a couple of the unit tests because of that I thought was a problem with certain events not being emitted by FS (in this case RECORD_STOP). After further investigation as part of #49 and #50 I am instead convinced that the problem is actually in using uuid_broadcast.

I believe this command is not working reliably (at least the same way it used to) and that parked sessions are not executing new dialplan apps requested using this method.

uuid_broadcast was only used initially due to switchy's dependence on the SWIG connection client (which we've since discarded) and not having an async way to invoke dialplan apps (i.e. there was no async way to support sendmsg). Now that we control the lower inbound protocol it makes to add this API support and stop using uuid_broadcast which is just a wrapper around what sendmsg would be doing anyway.

@moises-silva I'd appreciate your feedback on this as well!

Can't get variable through getvar() function -> `Session.getvar()` should return a `Future`.

Hi!
I'm trying to get a variable through * .getvar() and everything ends in an error.
My code:

@coroutine('CHANNEL_PARK')
async def on_park(self, sess):
    sess.setvar("test_var", "test_val")
    print(sess.getvar("test_var"))

Traceback:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/switchio/loop.py", line 49, in handle_result
    task.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 293, in result
    raise self._exception
  File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "./ontime/daemons/robot_esl_server_async.py", line 360, in on_park
    print(sess.getvar("test_var"))
  File "/usr/local/lib/python3.5/dist-packages/switchio/models.py", line 226, in getvar
    val = self.con.cmd("uuid_getvar {} {}".format(self.uuid, var))
  File "/usr/local/lib/python3.5/dist-packages/switchio/connection.py", line 246, in cmd
    event = self.api(cmd, block=True)
  File "/usr/local/lib/python3.5/dist-packages/switchio/connection.py", line 235, in api
    block=block,
  File "/usr/local/lib/python3.5/dist-packages/switchio/connection.py", line 53, in run_in_order_threadsafe
    future.result(timeout)
  File "/usr/lib/python3.5/concurrent/futures/_base.py", line 407, in result
    raise TimeoutError()
concurrent.futures._base.TimeoutError

The request for receiving is not visible in the FreeSwitch console...

Add pypy3 to CI

I know I had a problem before with pypy3 - it was horrendously broken on travis.
Hopefully we can add that back in and claim full pypy support!

Session API methods should be awaitable

@vodik made a good point during discussion of #34 that we should support both

await sess.answer()

and,

sess.answer()
await sess.recv("CHANNEL_ANSWER")

Such that sess.answer() == sess.recv("CHANNEL_ANSWER") is True.

Thread safety check in Future.add_done_callback()?

As per discussion with @vodik in friends-of-freeswitch/switchy#4, there seems to be a weird requirement when calling BaseEventLoop.call_soon() (called from Future.add_done_callback()) which has a thread id check only in debug mode.

Not sure if it's actually necessary and if we should patch the stdlib?
I added a FIXME as a reminder.

Use __slots__ on models

Thanks to @hockeybuggy for his great pycon Canada talk on this!
I think this can be used to slightly speed up some repetitive lookups on Session and Events objects.

Add statsd support

Issue by tgoodlet
Tuesday Mar 15, 2016 at 01:08 GMT
Originally opened as sangoma/switchy#30


Integrating with pystatsd will be the place to start.
This will give the RMS team something useful for monitoring real-time events from FreeSWITCH.

Change default call_id_var and app_id_var

Issue by tgoodlet
Wednesday Mar 16, 2016 at 03:49 GMT
Originally opened as sangoma/switchy#31


When stress testing external VoIP software, switchy attempts to track calls through an association of sessions (SIP legs) corresponding to each full media path. Having access to all sessions composing a call allows test apps to implement various higher level control logic. By default we have been appending X-headers using the outbound originate command (and thus SIP invite) which can be referenced on subsequent SIP legs/sessions known to FreeSWITCH. In other words, if the device under test (DUT) forwards custom headers then call tracking is supported by default. However, call tracking can be accomplished by using other SIP headers as is the case with stress testing our Vega product line.

Currently by default we:

  • track calls using a sip_h_X-switchy_originating_session header which should contain a unique id within the context of the test session / outbound calls set
  • activate switchy apps through a sip_h_X-switchy_app header which normally contains whatever name (str) the app was registered with

I propose instead to:

  • track calls by inserting a unique id into the FROM header's display name

    This is a good alternative to the current solution since it has nearly the same semantics (using a caller's "id"-ish header to identify unique calls).

  • activate switchy apps using the request-uri user part

    This falls inline with traditional notions of DIDs and extensions (i.e. routing semantics) when switchy is utilized as a call routing / dialplan alternative

I'm looking for any criticisms or foresight as to whether to go ahead with this @moises-silva, @ncorbic, @joegen.

Add Originator.unload_app

Issue by tgoodlet
Thursday Mar 17, 2016 at 23:33 GMT
Originally opened as sangoma/switchy#33


If you can load em you should be able to unload too. This is especially handy to have in pytest fixtures which preload smoke test apps prior to full on stress testing sessions.
Will most likely require patching the underlying AppManager.

High Availability of switchio itself?

Hi Guys,

Just discovered this project and really like the look of it, and the documentation is excellent.

I will more than likely move over to it, as the language/syntax looks really nice, however before/whilst I do I'm hoping you could answer a question for me.

Whilst switchio can be used to control a FreeSWITCH cluster, I'm wondering how can you make switchio itself highly available, or clustered?

I have a need to build a highly available solution in FreeSWITCH, with programmable call control, and what I have currently built is:

  • FreeSWITCH cluster (PostgreSQL in the core), with Keepalived, and can failover calls with 1-4 seconds of lost audio depending on the type of outage.
  • Outbound ESL connection in dialplan to my Python server running socketserver, native ESL (using swig), and can manage the calls as desired.

I am able to failover FreeSWITCH back and forth repeatedly with no issue, however the socket is broken, and so too is my ability to manage the call.

Do you have a solution, or idea, how this would work with switchio (or switchy)?
I presume I cannot have multiple switchio instances with the same config pointing to the same FreeSWITCH servers, as they with both want to manage the call simultaneously?

Another option might be to use VMware Fault Tolerance (limited to a single vCPU from memory, so won't scale well), any idea how switchio will behave if I 'sofia recover' the calls to another FreeSWITCH server?

I will eventually test this last one myself, and when I do report back, if I haven't heard anything back prior.

I know they're not easy questions and I appreciate you taking the time to read it.

Scaling with an actor model

@vodik pointed to quite impressive looking project pulsar which uses the same concurrency model as erlang: the actor model.

I think this is the correct solution to scale and add high-availability features to switchio.
I'm excited to try using it.

Update the ivr example for multiple apps

Issue by tgoodlet
Friday Jan 15, 2016 at 23:35 GMT
Originally opened as sangoma/switchy#24


The example ivr app should be updated to include multi-app originator usage.
That is using Originator.load_app() with a weight arg to show how to manage multiple outbound campaigns within a single process.

`waitfor` should unblock and raise an error on premature session hangup

Issue by tgoodlet
Thursday Aug 27, 2015 at 02:53 GMT
Originally opened as sangoma/switchy#12


Found an interesting isssue today where a test was waiting on the EventListener.waitfor call using the DtmfChecker app. The app in fact was not being invoked due to a failure with session-to-call association (due to the DUT not supporting x-header passthrough) and resulted in the waited on session actually being hungup much before the waitfor timeout. Really, if the session you're waiting on for a state change is hungup before any such change, an error should be raised immediately!

The solution will most likely include always setting events (if available) for waiters of a particular session on hangup here: https://github.com/sangoma/switchy/blob/master/switchy/observe.py#L586 by simply adding a second condtion:

if model.vars.get(varname) or model.hungup:
    map(Event.set, events)

In the latter case the waitfor logic will need to be adjusted to check for the hangup flag as well and raise an appropriate error in the calling thread somewhere here: https://github.com/sangoma/switchy/blob/master/switchy/observe.py#L633

My only concern would be for the case where some waiter is actually waiting for the hangup event itself. In that case I'm not sure exactly what to do... Maybe add a no_err_on_hangup flag to waitfor??

Allow running event loop in main thread

@vodik made another good point that we should support running event loops without a background thread. The user of the loop should be able to make the decision whether or not it should be run blocking. Support for this should be carefully considered as the eventual plan is to support multi-core via asyncio loops using multiprocessing.

I propose supporting this depending on how EventLoop is run/started (say via a block kwargs).
This also drums up questions about whether separate connect() and start() methods are even needed. The original motivation was to allow for event unsubscription via unsubscribe().

Not able to change the duration

Issue by UjjwalaVuyyala
Thursday Nov 02, 2017 at 16:13 GMT
Originally opened as sangoma/switchy#70


Hello,
Switchy is a great and easy to set up tool. I want to make calls to a particular URI and hold the call for 10 seconds and hangup. I am using the cli statement "switchy dial 127.0.0.1 --dest-url [email protected]:5060 --profile external --rate 1 --limit 1 --max-offered 5 --duration 10". The calls connect but a BYE is sent immediately ( within approx. .22 sec after the INVITE) no matter how much I change the rate, limit, max-offered and duration values. Any help to solve this issue is appreciated. Thank you in advance.

Rewrite ESL.py in python or cython

Issue by tgoodlet
Wednesday Feb 03, 2016 at 02:48 GMT
Originally opened as sangoma/switchy#26


I have suspicions that even if/when we move to mod_amqp it might not be as performant as plain old ESL for load testing. Either way, it would be great to not depend on the SWIG module and have a standalone python package.

This will mostly likely required a read of the libesl implementation to get going.

Call not hangup after playback done

I was trying the example:-

from switchio.apps.routers import Router

router = Router(guards={
    'Call-Direction': 'inbound',
    })

@router.route('(.*)')
async def welcome(sess, match, router):
    """Say hello to inbound calls.
    """
    await sess.answer()  # resumes once call has been fully answered
    sess.log.info("Answered call to {}".format(match.groups(0)))

    sess.playback('media.mp3') # non-blocking
    sess.log.info("Playing welcome message")
    await sess.recv("PLAYBACK_STOP")

    await sess.hangup()  # resumes once call has been fully hungup

After the media playback was done, the call just stay there and didn't hangup. After hanging up from the caller, I got this:-

07:05:31 ~/swio$ switchio serve 127.0.0.1 --app ./dialplan.py:router --password=cluecon
Dec 02 07:05:34 (MainThread) [WARNING] root storage.py:23 : No module named 'pandas'
Dec 02 07:05:34 (MainThread) [INFO] [email protected] loop.py:182 : Connected event loop 'c13c11e8-d6e3-11e7-adb4-06f257f4adeb' to '127.0.0.1'
Dec 02 07:05:34 (MainThread) [INFO] [email protected] api.py:145 : Loading 'Router' app with group id 'default' for event_loop '<switchio.loop.EventLoop object at 0x7fe01c9f69b0 [connected]>'
XXXXX CHANNEL_CREATE
XXXXX CHANNEL_PARK
XXXXX CHANNEL_ANSWER
Dec 02 07:05:43 (switchio_event_loop[127.0.0.1]) [INFO] switchio.str@unknown-host dialplan.py:12 : Answered call to ('60180044545',)
Dec 02 07:05:43 (switchio_event_loop[127.0.0.1]) [INFO] switchio.str@unknown-host dialplan.py:15 : Playing welcome message
XXXXX CHANNEL_HANGUP
Dec 02 07:06:46 (switchio_event_loop[127.0.0.1]) [WARNING] [email protected] loop.py:336 : Cancelling PLAYBACK_STOP awaited <Future pending cb=[Session.unreg_tasks(), <TaskWakeupMethWrapper object at 0x7fe01c1ea708>()]>
Stack for <Task pending coro=<Router.on_park() running at /home/kamal/miniconda3/lib/python3.6/site-packages/switchio/apps/routers.py:151> wait_for=<Future pending cb=[Session.unreg_tasks(), <TaskWakeupMethWrapper object at 0x7fe01c1ea708>()]> cb=[handle_result(log=<Logger switc....0.0.1 (INFO)>, model=<switchio.mod...-1559c2c9995a>)() at /home/kamal/miniconda3/lib/python3.6/site-packages/switchio/loop.py:45]> (most recent call last):
  File "/home/kamal/miniconda3/lib/python3.6/site-packages/switchio/apps/routers.py", line 151, in on_park
    await func()
Dec 02 07:06:46 (switchio_event_loop[127.0.0.1]) [ERROR] switchio.Router@['127.0.0.1'] routers.py:161 : Failed to exec <function welcome at 0x7fe01c1cd378> on match '601800818638' for session a41f4956-f556-4b41-9f03-1559c2c9995a
Traceback (most recent call last):
  File "/home/kamal/miniconda3/lib/python3.6/site-packages/switchio/apps/routers.py", line 151, in on_park
    await func()
  File "./dialplan.py", line 16, in welcome
    await sess.recv("PLAYBACK_STOP")
concurrent.futures._base.CancelledError
^CDec 02 07:07:02 (MainThread) [INFO] [email protected] loop.py:417 : Disconnecting event loop 'c13c11e8-d6e3-11e7-adb4-06f257f4adeb' from '127.0.0.1'
XXXXX SERVER_DISCONNECTED
Dec 02 07:07:02 (switchio_event_loop[127.0.0.1]) [WARNING] switchio utils.py:202 : Event 'SERVER_DISCONNECTED' has no timestamp!?
Dec 02 07:07:02 (switchio_event_loop[127.0.0.1]) [WARNING] [email protected] handlers.py:157 : Received DISCONNECT from server '127.0.0.1'

I print the evname in loop.py:_listen_forever() and we can see, no PLAYBACK_STOP coming. I was connected through telnet in the other console and I can see PLAYBACK_STOP coming to that connection so the event was fired by freeswitch.

Add a timeout to the example dialplan

One issue I noticed with park-only dialplan is that if for some reason the inbound socket process fail or die, then the parked calls will remained in freeswitch db (show calls) will list out the calls, filling out freeswitch internal db until it max out and freeswitch not accepting any new calls.

Is there something can be done in the dialplan other than just calling park, so that if there's no inbound server to process the call, it will just terminate the channel similar to outbound socket ?

Add bench marking tools/tests

Issue by tgoodlet
Friday Dec 18, 2015 at 20:22 GMT
Originally opened as sangoma/switchy#20


We need a reliable and consistent way to measure both python performance as well as FS performance as a load generator. A few approaches I can think of off-hand include:

  • include cProfile wrapper scripts much like pytest does: https://github.com/pytest-dev/pytest/tree/master/bench for bench marking:
    • event loop
    • measurement collection
    • call generator burst loop
  • create a test case which launches a SIPp uas with -rtp_echo enabled and run the Bert app to benchmark FS as a load generator.
    • Assuming SIPp can handle a lot of simultaneous RTP streams we should be able to find the breaking point of the FS caller.

better documentation

For a naïve user (myself), I have no idea what your software does. I do not know what "switchy" is and there is no link to that. A better written Readme.md with info on what your tool does would be helpful. I am guessing it can tall to FreeSWITCH using its event socket.

Document the `pandas` measurement sub-system

Issue by tgoodlet
Monday Dec 21, 2015 at 14:55 GMT
Originally opened as sangoma/switchy#21


This includes docs for the Measurers collection type, DataStorer and how it interfaces with measurement apps, and the AppManager.

We probably need:

  • example of how to write a measurement collection app
    • data storer insertion/api and app class attrs...
  • pandas.DataFrame schema description and processing/analysis examples
  • default figure walk-through with example
  • operators api
  • HDF storage and retrieval
  • figspec manipulation / extension

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.