udotdevelopment / atspm

Automated Traffic Signal Performance Measures (ATSPM) are a series of visual aids that display the high-resolution data from signal controllers. They are a valuable asset management tool, aiding personnel in the control of both signal hardware and signal timing and coordination. They allow analysis of data collected 24 hours a day, 7 days a week, improving the accuracy, flexibility, and performance of signal equipment and the system as a whole. ATSPM are used to optimize mobility, manage traffic signal timing and maintenance, reduce congestion, save fuel costs and improve safety.

ATSPM

Starting in 2002, Purdue University and Indiana DOT developed a partnership to evaluate the effectiveness of various vehicle detection devices. The result was a published standard in 2012 that defined traffic signal controller event codes, several key metrics, and charts that established the fundamental traffic signal performance measures. From this, Utah DOT developed this open source (Apache V. 2.0) software to automatically visualize these metrics and charts to show traffic engineers how their signals are performing in real-time and thus ATSPM was born. Data metrics include vehicle volumes, arrivals on red, pedestrian delay, split failures, and many more.

It's possible this software can be configured for other agencies to use.

TODO

Show how this software is deployed in a new setting

atspm's Issues

Watchdog alerts of all errors, not just the new ones

Theres a good reason UDOT doesn't want huge watchdog emails since they have so many signals. But some agencies may want all watchdog alerts to be included in a daily email, even if they are the same alerts from the previous day. If you want, here is a way to get all alerts everyday not just the new ones:

1194da5

The Website only displaying broken icons not the full chart

Hello everyone. I hope everyone is doing great. I am trying to install ATSPM website. I am now at step 2.7 ( create web service). However, when I am trying to create the default chart, the website is only showing the charts as broken Icon ( not the actual chart) ( like the following figure). But in the directory folder ( SPMImage folder), I can see the created images/metrics.
Do you have any idea, how to fix that issue/ why this is happening? Thanks in advance.

Incorporate code to ignore stop bar sensors, use only Approach Count

Arrivals on Red currently reads the stop bar and advance count sensors. This should look at advance counts only. Kittelson has written code to fix this and have posted it to OSADP.

ATSPM 4.2 Webconfig File

I was working on a new install of 4.2 by following the ATSPM 4.2 Installation Manual documentation. When editing the web.config file, the documentation calls for updating the ImageLocation, LinkPivotImageLocation, and SPMImageLocation keys. However, these are not included in the code anymore. Is this intentional or should it match the code from 4.0.1?

invalid IP

IP 0.0.0.0 will cause "FTPfromAllControllers" to crash

Unhandled Exception: System.AggregateException: One or more errors occurred. ---> System.ArgumentException: IPv4 address 0.0.0.0 and IPv6 address ::0 are unspecified addresses that cannot be used as a target address. Parameter name: address at System.Net.NetworkInformation.Ping.Send(IPAddress address, Int32 timeout, Byte[] buffer, PingOptions options) at System.Net.NetworkInformation.Ping.Send(String hostNameOrAddress, Int32 timeout, Byte[] buffer, PingOptions options) at FTPfromAllControllers.FTPfromAllControllers.CheckIfIPAddressIsValid(Signal signal) in c:\Projects\Branches\SignalPerformanceMetrics\v4.0.1\FTPfromasc3\Program.cs:line 195 at FTPfromAllControllers.FTPfromAllControllers.<>c__DisplayClass3.<Main>b__2(<>f__AnonymousType19 row) in c:\Projects\Branches\SignalPerformanceMetrics\v4.0.1\FTPfromasc3\Program.cs:line 105
at System.Threading.Tasks.Parallel.<>c__DisplayClass42_02.<PartitionerForEachWorker>b__1() at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask) at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object ) --- End of inner exception stack trace --- at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) at System.Threading.Tasks.Parallel.PartitionerForEachWorker[TSource,TLocal](Partitioner1 source, ParallelOptions parallelOptions, Action1 simpleBody, Action2 bodyWithState, Action3 bodyWithStateAndIndex, Func4 bodyWithStateAndLocal, Func5 bodyWithEverything, Func1 localInit, Action1 local Finally) at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable1 source, ParallelOptions parallelOptions, Action1 body, Action2 bodyWithState, Action3 bodyWithStateAndIndex, Func4 bodyWithStateAndLocal, Func5 bodyWithEverything, Func1 localInit, Action1 localFinally) at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable1 source, ParallelOptions parallelOptions, Action1 body) at FTPfromAllControllers.FTPfromAllControllers.Main(String[] args) in c:\Projects\Branches\SignalPerformanceMetrics\v4.0.1\FTPfromasc3\Program.cs:line 166

Getting Error doing The Watch dog setting V4.2

Hi All,
Doing this setup for Nebraska Dot , I looked on the documentation of WatchDog and it refrenced that the watchdog email need to be added on the Task config file , but I figured out that most of new version stuff has moved inside the site instead of the config files, so I logged as an admin and > ADMIN>APPLICATION SETTINGS > WATCHDOG setting and I get the below error :

Compilation Error
Description: An error occurred during the compilation of a resource required to service this request. Please review the following specific error details and modify your source code appropriately.

Compiler Error Message: CS1061: 'MOE.Common.Models.WatchDogApplicationSettings' does not contain a definition for 'EmailAllErrors' and no extension method 'EmailAllErrors' accepting a first argument of type 'MOE.Common.Models.WatchDogApplicationSettings' could be found (are you missing a using directive or an assembly reference?)
 

Line 104: Line 105:

Line 106: @Html.LabelFor(model => model.EmailAllErrors, htmlAttributes: new { @Class = "control-label col-md-2" }) Line 107:

Line 108:

c:\windows\system32\inetsrv> "C:\Windows\Microsoft.NET\Framework64\v4.0.30319\csc.exe" /t:library /utf8output /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\ea2fc76a\00d87768_e793d401\AlexPilotti.FTPS.Client.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\d9ed4a4a\00d87768_e793d401\Microsoft.AspNet.Identity.EntityFramework.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_64\System.EnterpriseServices\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.EnterpriseServices.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\Microsoft.Web.Infrastructure\v4.0_1.0.0.0__31bf3856ad364e35\Microsoft.Web.Infrastructure.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\819ce9d0\00d87768_e793d401\System.Web.Optimization.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\5ed51041\00d87768_e793d401\NuGet.Core.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\81f0a545\00d87768_e793d401\WebGrease.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\00e10da7\00d87768_e793d401\CsvHelper.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\d8b99f2e\00d87768_e793d401\EntityFramework.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\Microsoft.CSharp\v4.0_4.0.0.0__b03f5f7f11d50a3a\Microsoft.CSharp.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\66126a07\00d87768_e793d401\Caseiro.Mvc.PagedList.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Xml\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.ServiceModel\v4.0_4.0.0.0__b77a5c561934e089\System.ServiceModel.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Web.Extensions\v4.0_4.0.0.0__31bf3856ad364e35\System.Web.Extensions.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\a304e9f0\00d87768_e793d401\Microsoft.Extensions.Caching.Memory.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\0dc371e4\00d87768_e793d401\Microsoft.Threading.Tasks.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.ServiceModel.Web\v4.0_4.0.0.0__31bf3856ad364e35\System.ServiceModel.Web.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\e22537b5\00d87768_e793d401\Microsoft.Owin.Host.SystemWeb.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\9edfb80b\00d87768_e793d401\System.Web.Razor.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Web.ApplicationServices\v4.0_4.0.0.0__31bf3856ad364e35\System.Web.ApplicationServices.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\mscorlib.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\b9fd2fc5\00d87768_e793d401\Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.WorkflowServices\v4.0_4.0.0.0__31bf3856ad364e35\System.WorkflowServices.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Drawing\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Drawing.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\App_global.asax.7yzrhx9m.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\6cfa69dc\00d87768_e793d401\Microsoft.Owin.Security.MicrosoftAccount.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Web.Services\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.Services.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\98f6bfc0\00d87768_e793d401\System.Web.Mvc.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\85149c70\00d87768_e793d401\Microsoft.Owin.Security.Cookies.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\d0e7dfb4\00d87768_e793d401\Microsoft.Extensions.DependencyInjection.Abstractions.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\6ba43bfb\00d87768_e793d401\Microsoft.Extensions.Logging.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\30ef70fa\00d87768_e793d401\MvcCheckBoxList.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Configuration\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Configuration.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\19651cec\00d87768_e793d401\Microsoft.Owin.Security.OAuth.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Runtime\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Runtime.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\3d58f2ec\00d87768_e793d401\Microsoft.Extensions.Primitives.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\9a9431b8\00d87768_e793d401\System.Web.WebPages.Razor.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\1fb2e6b0\00d87768_e793d401\Microsoft.Extensions.Options.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\7b441dfc\00d87768_e793d401\SharpSnmpLib.Portable.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_64\System.Web\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\9ce75c75\00d87768_e793d401\Microsoft.Owin.Security.Google.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\f04fff1e\00d87768_e793d401\Microsoft.Extensions.Logging.Abstractions.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\788d72da\00d87768_e793d401\Microsoft.Extensions.Caching.Abstractions.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\af044a57\00d87768_e793d401\Microsoft.Threading.Tasks.Extensions.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\e633cc74\00d87768_e793d401\System.Web.Helpers.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\b62fd42a\00d87768_e793d401\Microsoft.Owin.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\bab88288\00d87768_e793d401\Microsoft.AspNet.Identity.Owin.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Data.DataSetExtensions\v4.0_4.0.0.0__b77a5c561934e089\System.Data.DataSetExtensions.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.ServiceModel.Activities\v4.0_4.0.0.0__31bf3856ad364e35\System.ServiceModel.Activities.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\26824020\00d87768_e793d401\DotNetZip.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\0b083415\00d87768_e793d401\System.Interactive.Async.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Runtime.Serialization\v4.0_4.0.0.0__b77a5c561934e089\System.Runtime.Serialization.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\a6fe2c9c\00d87768_e793d401\Remotion.Linq.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\f1e62396\00d87768_e793d401\EntityFramework.SqlServer.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Web.WebPages.Deployment\v4.0_2.0.0.0__31bf3856ad364e35\System.Web.WebPages.Deployment.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.ServiceModel.Activation\v4.0_4.0.0.0__31bf3856ad364e35\System.ServiceModel.Activation.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\90730d5b\00d87768_e793d401\Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\1f0ac8c5\00d87768_e793d401\Microsoft.Owin.Security.Facebook.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\10bfebe3\00d87768_e793d401\System.Web.WebPages.Deployment.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\4e806eee\00d87768_e793d401\SPM.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\300fd6c3\00d87768_e793d401\Microsoft.EntityFrameworkCore.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\3514cbf6\00d87768_e793d401\Microsoft.Web.XmlTransform.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Xml.Linq\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.Linq.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Web.DynamicData\v4.0_4.0.0.0__31bf3856ad364e35\System.Web.DynamicData.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Core\v4.0_4.0.0.0__b77a5c561934e089\System.Core.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\0c3322e6\00d87768_e793d401\System.Collections.Immutable.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\1412b55f\00d87768_e793d401\Microsoft.Owin.Security.Twitter.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Activities\v4.0_4.0.0.0__31bf3856ad364e35\System.Activities.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.ComponentModel.DataAnnotations\v4.0_4.0.0.0__31bf3856ad364e35\System.ComponentModel.DataAnnotations.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\cb64b185\00d87768_e793d401\FluentFTP.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\4fde9cc0\00d87768_e793d401\Microsoft.Threading.Tasks.Extensions.Desktop.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\40673cf6\00d87768_e793d401\Owin.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\577e9791\00d87768_e793d401\Antlr3.Runtime.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\f7ec3deb\00d87768_e793d401\SharpSnmpLib.Full.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\4245bf34\00d87768_e793d401\Microsoft.Extensions.DependencyInjection.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\68ef9aa5\00d87768_e793d401\Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\d417493d\00d87768_e793d401\MOE.Common.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\bd03e4aa\00d87768_e793d401\System.Web.WebPages.dll" /R:"C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.IdentityModel\v4.0_4.0.0.0__b77a5c561934e089\System.IdentityModel.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\a88a6db8\00d87768_e793d401\Microsoft.Owin.Security.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\375faec5\00d87768_e793d401\Newtonsoft.Json.dll" /R:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\assembly\dl3\357f7e64\00d87768_e793d401\Microsoft.AspNet.Identity.Core.dll" /out:"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\App_Web_edit.cshtml.d5158248.un29yhav.dll" /D:DEBUG /debug+ /optimize- /w:4 /nowarn:1659;1699;1701;612;618 /warnaserror- "C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\App_Web_edit.cshtml.d5158248.un29yhav.0.cs" "C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\atspmqa\8fa7fa7d\fa060d91\App_Web_edit.cshtml.d5158248.un29yhav.1.cs" Microsoft (R) Visual C# Compiler version 4.6.1586.0 for C# 5 Copyright (C) Microsoft Corporation. All rights reserved. This compiler is provided as part of the Microsoft (R) .NET Framework, but only supports language versions up to C# 5, which is no longer the latest version. For compilers that support newer versions of the C# programming language, see http://go.microsoft.com/fwlink/?LinkID=533240 i:\NDOTQA02\ATSPMQA\Views\WatchDogApplicationSettings\Edit.cshtml(106,39): error CS1061: 'MOE.Common.Models.WatchDogApplicationSettings' does not contain a definition for 'EmailAllErrors' and no extension method 'EmailAllErrors' accepting a first argument of type 'MOE.Common.Models.WatchDogApplicationSettings' could be found (are you missing a using directive or an assembly reference?) i:\NDOTQA02\ATSPMQA\Views\WatchDogApplicationSettings\Edit.cshtml(109,48): error CS1061: 'MOE.Common.Models.WatchDogApplicationSettings' does not contain a definition for 'EmailAllErrors' and no extension method 'EmailAllErrors' accepting a first argument of type 'MOE.Common.Models.WatchDogApplicationSettings' could be found (are you missing a using directive or an assembly reference?) i:\NDOTQA02\ATSPMQA\Views\WatchDogApplicationSettings\Edit.cshtml(110,59): error CS1061: 'MOE.Common.Models.WatchDogApplicationSettings' does not contain a definition for 'EmailAllErrors'

Can someone assist me with that, not sure what I am messing here.

Speed disappears after save unless Approach Speed is selected

The problem occurs when only Approach Count is selected

Purdue Phase Termination chart many unknown events

Siemens controller logs events at the same timestamp, and the way they're sorted by the software (in descending order) causes a lot of Unknowns (where event code 7 is immediately followed by another event code 7):

If you sort Ascending, then the unknowns don't happen:

Let me know if you would like further clarification

Command Timeout

While working to diagnose several exception errors I found that they all originated from the following error.

"Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding."

Since MnDOT compressed its data using ColumnStore the queries were failing due to there not being enough resources (CPU/RAM) before the default value for "CommandTimeout" in the database connection object expired.

Until more resources are allocated I worked around it by adding Database.CommandTimeout = 300;
between lines 18 and 19.

ATSPM/MOE.Common/Models/SPM.cs

Lines 15 to 24 in 9d25d14

	public SPM()
	: base("name=SPM")
	{
	Database.SetInitializer<SPM>(new CreateDatabaseIfNotExists<SPM>());
	}
	public static MOE.Common.Models.SPM Create()
	{
	return new MOE.Common.Models.SPM();
	}

Custom MOE Width

Is there a method for setting the size of the Metric Output Graphs? I would like to generate some 24hour graphs but given the multiple patterns the info at the top (Num of skips, Plan Num, etc.) gets cut off.

Split Monitor Metrics appears to be set to 740x500, Purdue Phase Termination 1100 x 550, etc.

SQL Username hard coded in migration

This SQL Server Agent Job looks useful, however it is hard coded to be owned by user 'SPM', so if your server doesn't have a user name 'SPM' it will make migrations fail in a non-obvious way with an error like

  System.Data.SqlClient.SqlException: The specified '@owner_login_name' is invalid (valid values are returned by sp_helplogins [excluding Windows NT groups]).

The workaround is create a user name SPM (you don't even have to change your connection string in the config file), or change this line in the code to match the username you're using in your connection string.

Hope this saves someone time.

ATSPM/MOE.Common/Migrations/ScheduleReclaimFileSpaceFirstFriday.sql

Line 27 in 87ab539

@owner_login_name=N'SPM', @job_id = @jobId OUTPUT

IIS/MS SQL alternatives...

I am just curious if there has been any discussion among the user community about porting development for use on a Linux nginx server with the dotnet core or using alternate database platforms such as PostgreSQL?

Obviously licensing is always an issue, so I'm just curious.

Metric Type Filter doesn't work for Turning Movement Counts

The "Metric Type" filter for "Turning Movement Counts" does not filter when selected. The "Metric Type" is on the front page under the "Signal Map" heading.

ApplicationEvents Table logs

While diagnosing connection issues I found that the FTPFromAllcontrollers is logging errors to the ApplicationEvents Table which does solve part of my question under #21 . Unfortunately there is no information on which signal each line refers too. Below is a summary of errors from 8 separate calls of the FTPFromAllcontrollers function. Some of them would appear to be random and just need to narrow down what signal it is and make sure the firmware is up to date. The log that is most concerning to me though is the "Transfer Task Timed Out." Are these normal? Should I be changing the Timeout values in the FTPFromAllcontrollers.config file ? I have changed the FTPFromAllcontrollers function to "non parrallel" while I am debugging these issues.

Any comments or suggestions would be welcome.

ApplicationName	FTPFromAllcontrollers
Total Logs	1437

DeleteFilesFromFTPServer	74
Cannot issue a new command while waiting for a previous one to complete	25
File "ECON_10.69.151.24_2017_12_12_0550.dat" not found or permission problem	8
Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.	34
Unable to read data from the transport connection: An established connection was aborted by the software in your host machine.	1
Unable to write data to the transport connection: An established connection was aborted by the software in your host machine.	6

ConnectToController	24
The connection task timed out for signal 10.69.140.51	8
The connection task timed out for signal 10.69.9.68	2
The connection task timed out for signal 10.69.31.158	2
The connection task timed out for signal 10.69.9.71	2
The connection task timed out for signal 10.80.69.45	1
The connection task timed out for signal 10.69.13.32	3
The connection task timed out for signal 10.69.158.15	1
The connection task timed out for signal 10.69.31.161	1
The connection task timed out for signal 10.69.9.19	1
The connection task timed out for signal 10.69.146.67	1
The connection task timed out for signal 10.69.9.11	1
The connection task timed out for signal 10.69.146.49	1

GetCurrentRecords	32
Invalid server reply:  SIZE	1
Not connected	8
Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.	19
USER and PASS required	4

RetrieveFiles	21
One or more errors occurred.	19
The operation was canceled.	2

TransferFiles	1286
Exception:The operation was canceled. While Transfering file: AlexPilotti.FTPS.Common.DirectoryListItem from //Set1 on 10.69.13.32 to C:\DownLoadedLogs\1736153\	47
Transfer Task Timed Out	1239

Link Pivot PCDs use random phases

This was caused by multiple approaches for the same direction. It would always take the first one set up in the database which in this case was number 5. Example from R2: Foothill, Should be Ph6.

FTP path hard coded

ATSPM/MOE.Common/Business/SignalFtp.cs

Line 81 in 067083b

if(!ftpClient.DownloadFile(SignalFtpOptions.LocalDirectory + Signal.SignalID+ @"\" + localFileName, ".."+Signal.ControllerType.FTPDirectory +@"/" + ftpListItem.Name))

This line is prepending ".." to the path. That might be a vendor specific thing so it should be in the FTPDirectory column of ControllerTypes in the database, not hardcoded here. This breaks controllers that don't use a relative path in their configuration.

Fix is remove ".."+ from the line above and it in the database instead.

Controller Event Log Size

My IT department has expressed concerns over the number rows in the Controller_Event_log table. Can anyone share there experience with backing up the table and/or performance issues querying the table.

Currently after roughly 3-4 weeks of data (~400 signals) it is at 50GB and 800 million rows.

Plot with start of interval, not end

On the charts that use Volume the data point is plotted at the end of the 15 minute period. This should be at the beginning of the 15 minute period.

ASC3 FTP - Don't retrieve latest file

Has there been any discussion about implementing a function to skip the most current file on the ASC3 to avoid breaking up the 15 min ".dat" files and thus not having to turn the high resolution logger on and off. I have noticed at several MnDOT signals we are missing 15 mins of data which I am assuming is caused by the ON/OFF procedure in that it received the OFF command but not an ON.

I see that on line 336 the code has a list of the remote files with .dat

ATSPM/MOE.Common/Business/Signal.cs

Line 336 in cbdea9d

if (RemoteFiles != null)

So I was thinking something along the lines of:

if  (RemoteFiles > 1)
  OrderbyName  //Not exactly sure how to code this yet
  SkipTopFile  //or this

Based on line 397 the code only deletes files in the list made on line 367 so the the current file should be left alone.

Thoughts?

SPM would be showing long service times in the split monitor for a phase for MaxTime controllers.

At least research this and see if it really is a problem we can do something about:

"In the attachment you will find the split monitor report from CDOT ATSPM which is based on the hi-resolution data retrieved from controller.
Could you please explain why phases have split length of approximately 400-500 sec for few cycles. Does this mean whenever controller goes into transition, the splits for the phases are that long?"

Approach Speed Y-axis min = 20, should be zero

Issue installing ATSPM 4.2

Hi all,
When installing ATSPM 4.2, I ran into the following error:

Unable to update database to match the current model because there are pending changes and automatic migration is disabled. Either write the pending model changes to a code-based migration or enable automatic migration. Set DbMigrationsConfiguration.AutomaticMigrationsEnabled to true to enable automatic migration. 

Do you have any suggestion on what to do?

Thanks

Set Username and Password Controller

Hi Dear MDoT:
I am installing ATSPM and I am on the step 2.8.1 "Install the 'FTPFromAllControllers'". We are using Siemens controllers and we usually retrieve HD data by using WinSCP. To access the data, we login through WinSCP with a username of "admin" and we do not have a password. Also, we have a private key file in order to connect to a controller.
On step 2.8.1.3, in SQL Server Management Studio, I edited the username as "admin" and the password as "none" and then run the executable. However, the connection failed. I was not sure if I should leave password blank since "none" also looked as a string. So I left the content of "password" blank and tried to connect the controllers again but it still failed.
So here is my question. Through all the steps, it did not ask me for the path of my "private key file", which is a necessity when we were using WinSCP to get connection to the controllers before. Will this be the reason that the connection could not be successfully made between my localhost and the controllers? If not, what could be the potential reasons? Thanks!

Incorporate code to fix Link Pivot problem with multiple detectors

Kittleson has fixed this issue in the code and posted it to the OSADP forum.

FTPfromAllControllers Log

Is there a built in tool for generating log files from the FTPfromAllControllers function?

"EntityValidationErrors"

When trying to create a chart I receive the message "Validation failed for one or more entities. See 'EntityValidationErrors' property for more details." Restarting the server resulted in the ability to create a few charts, but the error returns. Also, I've had this same message pop up when I already generated a metric and scrolled the page. I've done some online research and can not seem to find out a solution to this.

I am operating ATSPM version 4.0.1.

Thanks.

Version 4.2 Install MOE.Common Error

I was attempting a clean install of Version 4.2 and received the following error when trying to load the website for the first time. Will this same error impact the transition from 4.0 to 4.2?

"ApplicationEvent" table "Class" column string

When the program hits this point (if ever) it generates the string "MOE.Common.Models.Repositories.ControllerEventLogRepository" which is longer the the 50 character maximum defined in the "Class" column under the "ApplicationEvent" table

ATSPM/MOE.Common/Models/Repositories/ControllerEventLogRepository.cs

Line 212 in cbdea9d

e.Class = this.GetType().ToString();

Port Forwarding Solution - Signal SNMP and FTP Port

The SNMP port and FTP port in all functions should reference the signal table itself and not be hardcoded to a single number in the config files. These numbers are similar to an IP address and should be allowed to change from signal to signal and not be restricted to a single value within a controller type. The quick solution without rewriting a bunch of code was to just add more controller types for the additional cases. This however is not a clean solution.

ATSPM/FTPfromasc3/app.config

Line 19 in 21602f2

<add key="SNMPPort" value="161" />

Econolite Cobalt EOS

Not sure if anyone has used the ATSPM software on a Cobalt running EOS. It would appear that Econolite now renames complete files as ".datZ" and the current one is a ".dat"

The FTP process works fine and retrives them but the decoder does not. Not sure if you were going to make a separate decoder for Cobalts or not or wanted to keep it as part of the ASC3 Decoder.

ATSPM/NEWDecodeandImportASC3Logs/Program.cs

Line 79 in cbdea9d

SimplePartitioner<string> sp2 = new SimplePartitioner<string>(Directory.GetFiles(dir, "*.dat"));

Error Catching for Watchdog

From what I can tell the watchdog does not use FTP function and as such the line below appears to be an error.

ATSPM/SPMWatchDogNew/Program.cs

Line 44 in cbdea9d

errorRepository.QuickAdd("FTPFromAllcontrollers", "Signal", "TransferFiles", MOE.Common.Models.ApplicationEvent.SeverityLevels.High, ex.Message);

Errors when trying to FTP to controllers

Hi all,

After working my way through the setup process I finally have ATSPM running, the website is working, however whenever I try to make report the charts are all blank. When I try to FTP to all controllers (all econolite cobalts) it see lots of errors in the command prompt output, which is attached. It almost looks like the controller is rejecting the FTP connection?
spmFTPErrors.txt

Any help appreciated.

Signal Configuration Overlap won't save on first save

Single-digit detector channels will initially save as a 5-digit detector ID

After a few page refreshes or additional saves, it will be a 6-digit ID. Example: Signal # 7403 and adding detector channel 3. It shows up as 74033 instead of 740303 initially. I think I figured it out. When you add a detector and the default number is two digits but then you change it to a single digit detector, the initial DetectorID will only be 5 digits. In the attached example, I added Detector 10 and then saved. When I added the next number, the default value was 11. I changed that to 6 and saved. The Detector ID was not 11016 (it should have been 110106).

Server CPU/RAM requirements

Does anyone have a list of CPU/RAM requirements per signal?

I currently was allocated 4 CPU's (2.59GHz) and 8 GB of RAM and the system is struggling to retrieve and decode 400 signals in lest then 15mins. I could theoretically keep increasing these values since it is on a virtual machine but would like to know what others are running with to know what I should expect.

No endpoint listening at MetricGenerator.svc

I am still installing ATSPM v4.0.1 and I am now going on the step 2.7 "Create Web Services". I have finished the step from 2.7.a to 2.7.d. However, when I was trying on step 2.7.e by generating the charts on my localhost atspm website. The charts did not come out and there was an error message saying "There was no endpoint listening at http://mysite.com/MOEWcfService/MOEWcfServiceLibrary.MetricGenerator.svc that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details". I checked all the steps above and I was pretty sure I did not made major mistakes. So why does this problem happen and what should I do to solve it? I am looking forward to your reply and thanks!

Split Monitor Showing Partial Data

We are facing an issue with Split Monitor on 4.2.0.1 Version, Some Signals are showing Partial data for some phases Like Phase 1 and 5 Please check attached screenshots.

As per the Admin of the site the data is in the database and he verified that but not sure why it miss these time frame, any advice will be helpful

.

Siemens controller doesn't always let FTP user delete the logs from controller after import

Its a firmware problem that made the log files be owned by root and unable to be deleted by the admin account used to log in to FTP, and the FTP/Decoder would just re-import them, and end up with duplicate records in the ControllerEventLogs table. The site seems to still work with the duplicate events, but the database grows much faster, and probably is slower because of the extra logs. We only pull logs once per day so with 5 days of logs kept by default on the controller we had about 5x overhead in data, but if you pulled logs every 15 minutes and had this problem you'll end up with something like you data being duplicated up to 480 times.

Sample data for ATSPM

Hi all,
I've recently installed ATSPM in my laptop (with some support from this forum, thanks! :) )
I would like now to have some actual data in the DB that the installation process created. Maybe some simulated data so I can try the import tool?
I was wondering if that's something someone in this community could send me for test purposes.
Thanks much!
Juan

Unable to load embedded resource 'MOE.Common.Migrations.DropOrCompressTable.sql'

Hi I am installing the ATSPM ver 4.2 now and I have just finished the step 2.1. When I was trying to load my local website, the following error rouse(please see the attached picture). I just followed step by step according to the installation manual and had no clue what happened. Could I get some help about how to solve this? Thank you!

Link Pivot not working

Has anyone had any problems with getting the Link Pivot to work on the website? I am getting the PCD diagrams for the intersections in the route list, and the information shows up in the SQL server under ApproachRouteDetail. On the website, it shows the processing icon for about 1 second, then doesn't display anything.

The ApplicationEvents table in the SQL server is not showing any Link Pivot errors. The only error I get is "Transfer Task Timed Out", but usually doesn't affect anything as the data looks complete when running all of the other performance metrics. It just may have missed a upload, but got the upload at the next scheduled time.

The web.config is referencing the correct file paths, DNS server and endpoint addresses. All other performance metrics for the traffic signals work without issue.

Are there typical modifications to the LinkPivot code that anyone makes?

Data aggregation error

"Couldn't store in SignalId Column. Expected type is Int32. ---> System.FormatException: Input string was not in a correct format."

The database table SignalEventCountAggregations uses nvarchar for SignalId, but this code is looking for Int?. Our SignalIds have letters in them. Am I doing something wrong?
Full error below

Saving Signal Event Data to Database...

Unhandled Exception: System.AggregateException: One or more errors occurred. ---> System.ArgumentException: Input string was not in a correct format.Couldn't store in SignalId Column. Expected type is Int32. ---> System.FormatException: Input string was not in a correct format.
at System.Number.StringToNumber(String str, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean parseDecimal)
at System.Number.ParseInt32(String s, NumberStyles style, NumberFormatInfo info)
at System.String.System.IConvertible.ToInt32(IFormatProvider provider)
at System.Data.Common.Int32Storage.Set(Int32 record, Object value)
at System.Data.DataColumn.set_Item(Int32 record, Object value)
--- End of inner exception stack trace ---
at System.Data.DataColumn.set_Item(Int32 record, Object value)
at System.Data.DataRow.set_Item(DataColumn column, Object value)
at System.Data.DataRow.set_Item(String columnName, Object value)
at MOE.Common.Business.DataAggregation.DataAggregation.BulkSavePhaseTerminationData() in C:\Projects\GitHub\ATSPM\MOE.Common\Business\DataAggregation\DataAggregation.cs:line 613
at MOE.Common.Business.DataAggregation.DataAggregation.b__15_9() in C:\Projects\GitHub\ATSPM\MOE.Common\Business\DataAggregation\DataAggregation.cs:line 186
at System.Threading.Tasks.Parallel.<>c__DisplayClass4_0.b__0()
--- End of inner exception stack trace ---
at System.Threading.Tasks.Parallel.Invoke(ParallelOptions parallelOptions, Action[] actions)
at System.Threading.Tasks.Parallel.Invoke(Action[] actions)
at MOE.Common.Business.DataAggregation.DataAggregation.BulkSaveAllAggregateDataInParallel() in C:\Projects\GitHub\ATSPM\MOE.Common\Business\DataAggregation\DataAggregation.cs:line 100
at MOE.Common.Business.DataAggregation.DataAggregation.StartAggregation(String[] args) in C:\Projects\GitHub\ATSPM\MOE.Common\Business\DataAggregation\DataAggregation.cs:line 93
at AggregateATSPMData.AggregateAtspmData.Main(String[] args) in C:\Projects\GitHub\ATSPM\ArchiveMetricData\AggregateAtspmData.cs:line 26

Bulk Signal Import

MnDOT recently got its site running and would like to know if there is a method for importing the basic signal information (ID, Primary Name, Secondary Name, IP, Lat, Log, Region, Controller) in order to start acquiring event logs without having to go through the GUI for every signal? Can I insert them directly into the "Signals" table?

TMC ignores duplicate movement-lane configurations

Example, two channels are assigned to NBT Lane 1. Only the first lane will show. SIG#8153

The timer object in the FTP program is never renewed.

The timer object in the FTP program is never renewed. This means it will time out over slower connections.
We should renew the object after each successful file transfer.

Split Failure only looks back 30 minutes for Detector

Example: SIG#6054 on 2/22/2017, Ph7. Detector ON at 7:58:04 PM and next OFF at 9:41:07 PM. Split Failure shows only 30 minuets of Split Fails. Then it shows 0% GOR and ROR when it should show 100%. I'm assuming the algorithm only looks for detector events 30 m
Bug_SplitFail.pdf
inutes previous. See attached.

Canceling Create New Signal dialog creates (null) signal

This is a minor thing when using the new Online Signal Configuration tool. The link to 'Create New Signal' will create a signal with the specified ID if you click OK in the popup dialog box. However, if you click Cancel in that dialog it will still create a signal, but with null ID.

Why can't ATSPM Software Support Port forwarding?

Other central and or ATMS software programs support port forwarding. And if Udot's software is open source why can't the SQL server be configured to support Dynamic ports? Cell Modem use and Layer 3 switching. Is on the increase. In Traffic Network Deployments and they need to be supported. I think this the most practical solution.

Step 2.2 Launching the Website

I had downloaded the ATSPM 4.2 package from OSADP. When I was trying to launch the website, it failed and the web browser indicated that "A network-related or instance-specific error occurred while establishing a connection to SQL Server". I discussed with you in another issue #44 and did not find a reason for this error.

However, I was suspecting this error appeared due to that some of the files in OSADP package had not been updated yet. So I downloaded the newest release from Github. I deleted the IIS Manager and the website folder (C:\inetpub\wwwroot). Then I reinstall the IIS Manager and used the new ATSPM executable from Github. I thought this might work but it failed again. The browser showed "Parser Error". What could be the potential reasons for this? Thank you!

Edit the UDOT logo and embedded URL?

I would like to edit the UDOT logo and embedded URL found on the home page header to something else. What file can I adjust to change that image and URL? I couldn't find a file that had that content. I'm on ATSPM 4.2.