tfsaggregator / aggregator-cli

A new version of Aggregator aiming at Azure DevOps (ex Visual Studio Team Services)

aggregator-cli

Aggregator is a Rule Interpreter for Work Items events, allowing dynamic calculation of field values and more.

It is the successor to renowned TFS Aggregator. The current Server Plugin version (2.x) will be maintained to support TFS. The Web Service flavour will be discontinued in favour of this new tool for two reasons:

• deployment and configuration of Web Service was too complex for most users;
• both the Plugin and the Service rely heavily on TFS Object Model which is deprecated.

Aggregator 3.x supports two scenarios:

1. Azure DevOps Services with Rules running in Azure Functions.
2. Docker container running in the cloud or on-premise. [v1.0]

The latter permits replacing the Server Plugin after migrating the Rule code.

Note: This README is a synopsis of the documentation available at https://tfsaggregator.github.io/docs/v3/.

You might also find useful Richard Fennell's post Getting started with Aggregator CLI for Azure DevOps Work Item Roll-up.

Major features

• use of new Azure DevOps REST API
• simple deployment via CLI tool or Docker container
• Rule language similar to TFS Aggregator v2

Planned features (post v1.0)

• Support for Deployment Slots for blue/green-style deployments
• OAuth support to avoid maintain access tokens
• Additional Azure DevOps events and objects (e.g. Git)

Requirements

To write a Rule is required some basic knowledge of C# language and Azure Boards. In addition you need:

• an Azure DevOps Services Project
• a Personal Access Token with sufficient permissions on the Project

CLI & Azure

The CLI scenario has two additional requirements:

• an Azure Subscription
• a Service Principal with, at least, Contributor permission on a Resource Group of the Subscription

Docker

The Docker scenario requires:

• an SSL Certificate
• an host for Docker containers (Windows or Linux)

How the CLI works with Azure Functions

As the name implies, this is a command line tool: you download the latest aggregator-cli*.zip appropriate for your platform from GitHub releases and unzip it on your client machine. Read more below in the Usage section.

Through the CLI you create one or more Azure Functions in your Subscription. The Functions use a library named Aggregator Runtime to run your Rules. A Rule is code that reacts to one or more Azure DevOps event; currently, the only language for writing rules is C#.

The CLI automatically checks GitHub Releases to ensure that you use the more recent version of Aggregator Runtime available. To avoid automatic upgrades, specify the Runtime version or point to a specific Runtime package file, using an http: or file: URL.

After you setup the Rules in Azure, you must add at least one Mapping. A mapping is an Azure DevOps Service Hook that send a message to the Azure Function when a specific event occurs. Currently we support only Work Item events. When triggered, the Azure DevOps Service Hook invokes a single Aggregator Rule i.e. the Azure Function hosting the Rule code. Azure DevOps saves the Azure Function Key in the Service Hook configuration.

You can deploy the same Rule in different Instances, map the same Azure DevOps event to many Rules or map multiple events to the same Rule: you choose the best way to organize your code.

CLI Authentication

You must instruct Aggregator which credential to use. To do this, run the login.azure and login.ado commands.

To create the credentials, you need an Azure Service Principal and a Azure DevOps Personal Access Token. Full details in the Setup section.

Aggregator stores the logon credentials locally and expires them after 2 hours.

The PAT is also stored in the Azure Function settings: whoever has access to the Resource Group can read it!

The Service Principal must have Contributor permission to the Azure Subscription or, in alternative, pre-create the Resource Group in Azure and give the service account Contributor permission to the Resource Group. If you go this route, remember add the --resourceGroup to all commands requiring an instance.

For Docker only Azure DevOps logon is required.

CLI Usage

Download and unzip the latest CLI.zip file from Releases. It requires .Net Core 3.1 installed on the machine. To run Aggregator run aggregator-cli.exe (Windows), aggregator-cli (Linux) or dotnet aggregator-cli.dll followed by a verb and its options.

CLI Verbs

There are about 20 commands described in detail at Commands.

They can be grouped in a few categories:

• Authentication to logon into Azure and Azure DevOps.
• Instance creation, configuration and update.
• Rule deployment, configuration and update.
• Mapping from Azure DevOps to Rules.
• Informational commands, to read configuration.
• Testing commands to validate configuration.

Most commands manage Azure Function, but a few can be used in the Docker scenario.

We collected some usage scenarios at Command Examples.

How the Docker image works

Pull the latest image from Docker Hub. It works on Linux and Windows. Start a container with the image, setting configuration through environment variables. The Rules are simply files on a Docker volume that the container uses. The container must expose a port reachable from your Azure DevOps instance, either Server or Service. Add one or web hook to Azure DevOps using the container URL. Use one Aggregator API Key to authenticate the call. You may use the CLI to add these mappings.

More details at Docker configuration

Rule language

Currently we offer only C# as the language to write Rules. The Rules can access a few objects:

• The Current Work Item.
• Work Item Store to retrieve additional Work Items.
• The Event which triggered the Rule.
• Project information.
• A Logger object to track Rule steps.

See Rule Language for a list of objects and properties to use. For examples see Rule Examples.

Maintenance

Aggregator stores the PAT in the Azure Function configuration. Before the PAT expire you should refresh it from Azure DevOps or save a new PAT using the configure.instance command.

Read Production Configuration and Administration for recommendations on running Aggregator in production.

Troubleshooting

Use the Application Insight instance that was created aside the Azure Function.

Contributing

Details on building your own version and testing are in the Contribute section.

aggregator-cli's Issues

Change item status in another organization

I tried to change Azure DevOps item status in an organization A that has no Azure directory assigned. The used Azure directory belongs to a Microsoft user which is a user in organization A. I was able to setup an Aggregator instance and an Aggregator Rule in Azure. I was also able to setup an Aggregator mapping via cli in Azure DevOps. When triggered the Azure DevOps Service Hook invokes the Aggregator Rule with the following output: [Warning] Rule 'itemstatus' failed: VS30063: You are not authorized to access https://xxx... (organization A).
Is this scenario simply not supported or do I miss anything?

Error when mapping rule - AzureWebJobsSecretStorageType does not support blob in functions v2

Error thrown when trying to run map.rule

Failed to retrieve function key: {"Message":"An error has occurred.","ExceptionMessage":"Runtime keys are stored on blob storage. This API doesn't support this configuration. Please change Environment variable AzureWebJobsSecretStorageType value to 'Files'. For more info, visit https://aka.ms/funcsecrets","ExceptionType":"System.InvalidOperationException","StackTrace":" at Kudu.Core.Functions.FunctionManager.<GetKeyObjectFromFile>d__91.MoveNext() in C:\Kudu Files\Private\src\master\Kudu.Core\Functions\FunctionManager.cs:line 141\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Kudu.Core.Functions.FunctionManager.d__12.MoveNext() in C:\Kudu Files\Private\src\master\Kudu.Core\Functions\FunctionManager.cs:line 220\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Kudu.Services.Functions.FunctionController.d__12.MoveNext() in C:\Kudu Files\Private\src\master\Kudu.Services\Functions\FunctionController.cs:line 141"}`

unmap.rule lacks project parameter

The unmap.rule command does not allow team project to be specified like the map.rule does.

How to get values of Parent task in Child task...

Hi,
I want to get field values (e.g. DueDate of Parent task) into Child task, it is like when a child task will be created its DueDate will be Auto filled to same as Parent task dueDate is.
I am using following code for this but it is not working, it is giving me following error under Service Hook:

• 503 - Not Implemented
  Sample Code:

.lang=C#
using System;
using System.Collections;
using System.Collections.Generic;

string message = "";
var parent = self.Parent;
if (parent != null)
{
var children = parent.Children;
self["System.DueDate"] = parent["System.DueDate"];
}

Consider fail fast in CLI commands

in many commands the options are checked after connecting to Azure/Azure DevOps. If the check fails both connections were being made unnecessarily. It is beneficial to check them before accessing a different system.

Rule for customfield

I have new custom field created by name TestMethodID and when I am trying to push some values to it by creating simple rule like below I am getting error. Please help

Rule:
.language=Csharp
if (self.WorkItemType == "Test Case")
{
self.TestMethodID =1234;
}
return "Nothing to do";
Error:

'WorkItemWrapper' does not contain a definition for 'TestMethodID' and no accessible extension method 'TestMethodID' accepting a first argument of type 'WorkItemWrapper' could be found (are you missing a using directive or an assembly reference?)

After adding two or more rules error: "Select a language for your function app by specifying FUNCTIONS_WORKER_RUNTIME AppSetting."

After adding a second rule in a function instance the azure functions portal throws
"The function runtime is unable to start. Microsoft.Azure.WebJobs.Script: Found functions with more than one language. Select a language for your function app by specifying FUNCTIONS_WORKER_RUNTIME AppSetting."

Fix is adding setting to the instance
FUNCTIONS_WORKER_RUNTIME : dotnet

Accessing Azure DevOps Web API/non exposed functionality

Rational

For a rule I have to know the category of a work item's state. This information is currently not exposed.
E.g. the category of the state Done of the work item type Bug of the process template Scrum

To get this information I can

a) write a constant mapping in the source code. Then I have to type a lot and customized process templates are not supported.
b) query the web/Rest API. Then customized process templates are supported.

I favor a).

Question

How can accessing the web/Rest API achieved?
Answer: Something has to be implemented 😉

Possible Implementations

a) Write a wrapper and expose it via WorkItemStore

public class WorkItemStore
{
    public async Task<string[]> GetWorkItemStatesAsync(string workItemType) { /* ..*/ }
}

b) Write a wrapper calls and expose it global type
c) Provide the possibility to extend the functionality (of a rule) without adjusting Aggregator. Directly access the Web API from the rule

Verify rule before uploading

Given a rule containing (only a parentheses)

(

When adding the rule
and mapping the rule
Then no errors occurs

Actual

Upload

No error occurs

Runtime

When the rule is invoked
then following error occurs

 [Warning] Rule 'Invalid' failed: (1,2): error CS1733: Expected expression

Expected

The CLI should not allow to upload invalid rules (according to the rule language), an error should be produced

Relation navigators on WorkItemWrapper should return null if not present

Something like item.Parent will throw an exception if there is no parent linked work item. Should just return null for reference checking. Currently have to check the link version of the field before calling the primary.

item.Parent
item.Children

Support for non-expiring Azure DevOps credentials

Current version has only support for PAT, which can last, at most, a year.

Propose - Allow multiple App Function Instances per Resource Group

As is the application assumes the 'name' param of the instance.install command is both the resource group AND function application name. The proposal is to separate out the two params (Resource Group name and Function App Name) with app name being optional, if second param not provided then default to same value as RG name.

If the function app name param is used then install/update the function application under that name inside the provided resource group name.

Manage throttling

Are we using a Durable Function to ensure eventual delivery of notifications, even if the account trips a circuit breaker?
When we invoke a REST api we should get a requests header that tells us we're throttled or that the account is temporarily unavailable. If we use a durable function then we could sleep for a short while or we could also tell the function runtime to call again in a couple seconds to improve reliability of rule execution.

From https://docs.microsoft.com/en-us/azure/devops/integrate/concepts/rate-limits?view=azure-devops

When an individual user's requests are blocked, responses with HTTP code 429 (too many requests) will be received, with a message similar to the following:
TF400733: The request has been canceled: Request was blocked due to exceeding usage of resource <resource name> in namespace <namespace ID>.

Help needed to push some default text in to multi text field inside Azure Devops

Hello,

Whenever a new work item gets created want to push some default text into mult line text field area. Can this be done using CLI aggregator. If yes, can you please help me to with sample code

Thanks,
Prashanth

One or more errors occurred. (The subscription 'xxx' could not be found.)

Brand new; no instances created yet.

1. logon.azure succeeds.
2. logon.ado succeeds.
3. list.instances throws error for the same subscription it successfully connected to a moment prior:

One or more errors occurred. (The subscription 'xxx' could not be found.)

How to get values of Successor and Predecessor task's fields...

Hi,
How do I get successor and predecessor field values? Following is an example which I used to get Parent and Child tasks field values, but when I use Successor or Predecessor then it gives error.

var parent = self.Parent;
var child = self.Children;
child["Microsoft.VSTS.Scheduling.TargetDate"] =self["Microsoft.VSTS.Scheduling.TargetDate"];

As in above code assigning value of parent field to child field, how can I do the same for successor and predecessor task fields.
var successor = self.Successor; ---> This gives error
var successor = self.Successors; ---> This gives error
Error CS1061: 'WorkItemWrapper' does not contain a definition for 'Successor'

Thanks,
Kamran

Aggregator fail with not implemeted 501 statuscode - on premise Azure DevOps 2019

I have setup a function instance in Azure with one simple rule. In my script i also add webhooks on my on premise Azure Devops 2019 instance.

Everything works when i run a test webhook call but when i run a real hook on for example workitem.update i get this error.

I changed the CLI code to give me more information on error.

λ  dotnet $aggregatorCli stream.logs --instance $aggregatorName --resourceGroup $resourceGroup --verbose
[2019-08-19 10:35:00Z] Aggregator CLI v0.9.0+local.development (build: 0.9.0.0 Debug) (c) TFS Aggregator Team
[2019-08-19 10:35:00Z] Authenticating to Azure...
[2019-08-19 10:35:01Z] Connected to subscription 20bba462-f2a0-435e-adb7-4d0f8d775e23
[2019-08-19 10:35:01Z] Connecting to FunctionWI6...
[2019-08-19 10:35:06Z] Connected to FunctionWI6 logs
[2019-08-19 10:35:18Z] Streaming FunctionWI6 logs...
2019-08-19T08:35:18  Welcome, you are now connected to log-streaming service.
2019-08-19T08:35:19.038 [Information] Executing 'Functions.writeworkitemtypeandtitle' (Reason='This function was programmatically called via the host APIs.', Id=ea3e2242-0905-4da4-959b-738146178263)
2019-08-19T08:35:19.038 [Information] Executing 'Functions.writeworkitemtypeandtitle' (Reason='This function was programmatically called via the host APIs.', Id=ea3e2242-0905-4da4-959b-738146178263)
2019-08-19T08:35:19.920 [Information] Aggregator v0.9.1+local.development executing rule 'writeworkitemtypeandtitle'
2019-08-19T08:35:20.092 [Warning] collectionUrl https://tfs-test.xxx.com/tfs/Test_XXXCareCollection/
2019-08-19T08:35:20.093 [Information] teamProjectId b328f48c-4979-4d71-9090-bae5966771bf
2019-08-19T08:35:20.093 [Information] workItem Microsoft.TeamFoundation.WorkItemTracking.WebApi.Models.WorkItem
2019-08-19T08:35:19.920 [Information] Aggregator v0.9.1+local.development executing rule 'writeworkitemtypeandtitle'
2019-08-19T08:35:20.092 [Warning] collectionUrl https://tfs-test.raysearchlabs.com/tfs/Test_xxxCareCollection/
2019-08-19T08:35:20.093 [Information] teamProjectId b328f48c-4979-4d71-9090-bae5966771bf
2019-08-19T08:35:20.093 [Information] workItem Microsoft.TeamFoundation.WorkItemTracking.WebApi.Models.WorkItem
2019-08-19T08:35:21.047 [Warning] Rule 'writeworkitemtypeandtitle' failed:    at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Location.Client.LocationHttpClient.GetConnectionDataAsync(ConnectOptions connectOptions, Int64 lastChangeId, CancellationToken cancellationToken, Object userState)
   at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.ConnectAsync(ConnectOptions connectOptions, CancellationToken cancellationToken)
   at aggregator.RuleWrapper.ExecuteAsync(Uri collectionUri, Guid teamProjectId, WorkItem workItem, CancellationToken cancellationToken) in C:\git\aggregatorazuredevops\aggregator-cli\src\aggregator-function\RuleWrapper.cs:line 51
   at aggregator.AzureFunctionHandler.RunAsync(HttpRequestMessage req, CancellationToken cancellationToken) in C:\git\aggregatorazuredevops\aggregator-cli\src\aggregator-function\AzureFunctionHandler.cs:line 116
2019-08-19T08:35:21.088 [Information] Executed 'Functions.writeworkitemtypeandtitle' (Succeeded, Id=ea3e2242-0905-4da4-959b-738146178263)
2019-08-19T08:35:21.047 [Warning] Rule 'writeworkitemtypeandtitle' failed:    at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Location.Client.LocationHttpClient.GetConnectionDataAsync(ConnectOptions connectOptions, Int64 lastChangeId, CancellationToken cancellationToken, Object userState)
   at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.ConnectAsync(ConnectOptions connectOptions, CancellationToken cancellationToken)
   at aggregator.RuleWrapper.ExecuteAsync(Uri collectionUri, Guid teamProjectId, WorkItem workItem, CancellationToken cancellationToken) in C:\git\aggregatorazuredevops\aggregator-cli\src\aggregator-function\RuleWrapper.cs:line 51
   at aggregator.AzureFunctionHandler.RunAsync(HttpRequestMessage req, CancellationToken cancellationToken) in C:\git\aggregatorazuredevops\aggregator-cli\src\aggregator-function\AzureFunctionHandler.cs:line 116
2019-08-19T08:35:21.088 [Information] Executed 'Functions.writeworkitemtypeandtitle' (Succeeded, Id=ea3e2242-0905-4da4-959b-738146178263)
2019-08-19T08:35:22.829 [Information] Host lock lease acquired by instance ID 'bec43f7473ffa1a31147948b5c1b6b8e'.

Here is the test call that works

Here is the call from a real workitem

Here is the error in code in AzureFunctionHandler.cs

I also found on Azure function that Aggregator_VstsToken value is empty - isnt it here PAT token should be? I tried to put in my PAT token manually but it doesnt work either.

I tried to debug the code and see why is this happening but cannot fix it. :( Can you please give me some advice.

Permit choosing plan, Consumption or App Service and which of the latter

Current template uses free App Service plan, so one has to manually change using Azure Portal, CLI or Powershell for other options.

We should allow additional options on Microsoft.Web/serverfarms, in particular the Consumption plan, that is

"sku": {
    "name": "Y1",
    "tier": "Dynamic",
    "size": "Y1",
    "family": "Y",
    "capacity": 0
},

Error on new item w/ relation save "IdentityRef type is not accepted for this request."

Save failed: {"count":1,"value":{"Message":"VS403426: IdentityRef type is not accepted for this request."}}

RuleEngine.ExecuteAsync() always sets State to Success

Is the behavior of always succeeding regarding to RuleEngine.State on purpose? See line 114

aggregator-cli/src/aggregator-ruleng/RuleEngine.cs

Lines 101 to 114 in 3916231

	if (result.Exception != null)
	{
	logger.WriteError($"Rule failed with {result.Exception}");
	State = EngineState.Error;
	}
	else if (result.ReturnValue != null)
	{
	logger.WriteInfo($"Rule succeeded with {result.ReturnValue}");
	}
	else
	{
	logger.WriteInfo($"Rule succeeded, no return value");
	}
	State = EngineState.Success;

Function runtime unable to start

The function runtime is unable to start. Microsoft.Azure.WebJobs.Script: The host.json file is missing the required 'version' property. See https://aka.ms/functions-hostjson for steps to migrate the configuration file.

Allow casting of IdentityRef type

We need to be able to cast the IdentityRef field in rule code in order to interact with custom work item fields beyond the standard ones.

Support for TFS 2017 update 3 on prem - Non beta version

Hi,

It seems like the previous recommended version for TFS 2017 update 3 on prem is the web service version of the TFS Aggregator, and now it is the Aggregator-cli.
As we need to examine the aggregation solution for our organization, and prefer not using a beta version, can you please recommend which version we should try for POC?
Is the Release version of Aggregator-cli expected soon?

Thanks,
Sharon.

delete event fails

We are noticing in the log stream that the DELETE event still tries to retrieve the work item and fails since it no longer exists.

I don't think the rule even starts if the work item cannot be retrieved.

Thanks

aggregator-cli/src/aggregator-ruleng/RuleEngine.cs

Line 118 in 450d694

var self = store.GetWorkItem(workItemId);

Custom fields

SImilar to #58 which has been closed. It would be great if you could share how to get values from a custom field.

.lang=C#
var value = self['customField'];
return value;

Returns the following:

One or more errors occurred. ((4,8): error CS0266: Cannot implicitly convert type 'object' to 'string'. An explicit conversion exists (are you missing a cast?))

Be able to use "WorkItem" type in the definition of a rule

Be able to declare a WorkItem variable :
WorkItem _my;

WorkItemWrapper or aggregator.Engine.WorkItemWrapper doesn't work.

giuliov : "we need to add a few more assembly references in the RuleEngine constructor to make this work"

New task: Code in documentation does not work

This code throws exception, taken from documentation.

var parent = self;
var newChild = store.NewWorkItem("Task");
newChild.Title = "Brand new child";
parent.Relations.AddChild(newChild);

return parent.Title;

Wrapping up for 1.0

This issue tracks the discussion for the 1.0 release.

My (@giuliov) opinion on remaining issues:

• must be solved before release: #55, #45, #26
• nice to have, but can be postponed after 1.0: #62, #40, #35, #14

Am I missing anything important, @jessehouwing @BobSilent @pswetz @manne ?

Reason Phrase: Not Implemented

Hello,
I tried to create a simple test rule. And from Service hook Azure devops I am getting below error when I created new Bug.
Reason Phrase: Not Implemented
I just added a simple rule from examples and mapped that rule from azure to ADO.Rule I uploaded is $"Hello { self.WorkItemType } #{ self.Id } - { self.Title }!".
Please help. I am ready to provide more details in case.

Confusing CLI command requirements for resource group and location

The resource group parameter is not required, but the location parameter is required. As such, one would expect to be able to use any existing resource group and provide any location they desire. However, the supplied location argument is only used for new resource group creation, not the resources within (ex: app plan).

If a necessary resource is not available in the region of the resource group, the instance installation fails regardless of the input location.

In our case, all our resource groups are in WestUS. The deployment kept failing for Application Insights because the product is not available in the WestUS region. We had to dig through the code to figure out why - despite setting the instance location to WestUS2 - the CLI kept trying to use WestUS as the location for all the resources.

The application should be published as release

Expected
The solution should be build with the release configuration.

Actual
The solution is built with the debug configuration.

[command]"C:\Program Files\dotnet\dotnet.exe" build D:\a\1\s\src\aggregator-cli.sln
Microsoft (R) Build Engine version 15.8.166+gd4e8d81a88 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

Link

Explanation
the default value of dotnet build for the option configuration is Debug. See Docs dotnet build

Implementation
Append the option --configuration Release to the .NET Core build task named dotnet build in Azure Pipelines

Hint
If the YAML Azure Pipelines file is present in this repo, then I could create a PR for this matter. See Create your first pipeline

Edit: added link to build log

One or more errors occurred. (Operation returned an invalid status code 'Forbidden')

Hi everyone,

I succeed my authentication with logon.azure and logon.ado, but on install.instance
with a custom name, location and a new resourcegroup name as i understabd the latter is going to be standing beside my rg for my organisation and project.

While debugging i get this:

Exception thrown: 'Microsoft.Rest.Azure.CloudException' in System.Private.CoreLib.dll
Exception thrown: 'Microsoft.Rest.Azure.CloudException' in System.Private.CoreLib.dll
Exception thrown: 'System.AggregateException' in System.Private.CoreLib.dll

here

Could anyone suggest what I am doing wrong with this?

Thank you,

Tino

IdentifyRef field type: Unable to cast object of type 'Microsoft.VisualStudio.Services.WebApi.IdentityRef' to type 'System.String'.

Unable to cast object of type 'Microsoft.VisualStudio.Services.WebApi.IdentityRef' to type 'System.String'. at aggregator.Engine.WorkItemWrapper.GetFieldValue[T](String field) in D:\Repos\aggregator-cli\src\aggregator-ruleng\WorkItemWrapper.cs:line 412at aggregator.Engine.WorkItemWrapper.get_AssignedTo()

Then when trying to case like
string assigned = ((Microsoft.VisualStudio.Services.WebApi.IdentityRef)self["AssignedTo"])?.DisplayName ?? "";

throws
type or namespace name 'VisualStudio' does not exist in the namespace 'Microsoft'

Need update
public IdentityRef AssignedTo

    {

        get { return GetFieldValue<IdentityRef>(CoreFieldRefNames.AssignedTo); }

        set { SetFieldValue(CoreFieldRefNames.AssignedTo, value); }

    }

and also Roslyn reference updates for types.

Support for ApplicationInsights

Permit logging on AI as alternative to built-in logging.

Reason

The built-in logging is useful for testing with light workloads but is not intended for high-load production use. For production monitoring, Application Insights is recommended. If built-in logging is used in production, the logging record may be incomplete due to throttling on Azure Storage.
https://docs.microsoft.com/en-us/azure/azure-functions/functions-monitoring#disable-built-in-logging

Foreseeable changes
ARM template creates AI resource
Configuration command to switch between modes by removing adding these keys:

• AzureWebJobsDashboard connection string to storage created by template
• APPINSIGHTS_INSTRUMENTATIONKEY key of AI
  Do not forget to expose logging to rules

Open issues
One AI resource per Instance or one per RG or what?

Issue with rollup type rules when PBI/bug only has 1 child task

I added a simple rollup rule to calculate Completed Work / Original Estimate on PBI/bug. This works for PBI's and bugs which have multiple child tasks but when there's only 1 child it fails. Failures return 501 Not Implemented with error "The given key 'False' was not present in the dictionary.". I think the Children relation is the issue.

Here's my rule for Original Estimate:
if (self.Parent != null)
{
var parent = self.Parent;
parent["Microsoft.VSTS.Scheduling.OriginalEstimate"] = parent.Children.Sum(task => Convert.ToDouble(task["Microsoft.VSTS.Scheduling.OriginalEstimate"]));
}

Random Error: Log is being used by another process

Not consistent error, mostly seems to happen after an initial function app warmup but is inconsistent.

[Warning] Rule '[RuleName]' failed: An error occurred while writing to logger(s). (The process cannot access the file 'D:\home\LogFiles\Application\Functions\Function[RuleName]\2018-11-19T20-41-00Z-60d3d534f7.log' because it is being used by another process.)

Let me specify a different region for Application Insights

Application Insights is automatically provisioned as part of the creation of an instance. However, Application Insights is not available in all Azure datacenters, eventually leading to a failed deployment when deploying to e.g. Brazil South.

I want to be able to deploy my Aggregator instance closer to my Azure DevOps instance (e.g. deploying to Brazil South when connecting to Azure DevOps orgs hosted in Brazil), while at the same time deploying App Insights wherever it's available.

Apply some generic rules at organization level to all projects

Hi,
Is there any way we can apply a common rule at organization level to all projects under that organization and also when any new project created under that organization then that rule by default applies to the new project too. Following is example:

• Whenever a work item created, it's CreateDate field should be auto filled to currentdate. This rule should be default to each project under an organization and also when a new project created this rule should be auto apply so that I do not have to run:
  aggregator-cli map.rule
  command against each project in the organization.
  The above example is simple, it is just to make my question simple. I want some rules to apply at organization level and whenever user creates project they should auto apply to those projects. Is it possible?

On Update Events, is it possible to update a work item as the previous committer rather than the service hook owner?

In my example, when a rule for created/updated is activated, it make a change to a field in the current work item. This shows up in the history graph as expected but as executed by the Service Hook Owner.

Is it possible to set the Changed By for the rule to the person who saved the Work Item ?

Thanks
Dan

uninstall.instance deletes entire Resource Group even if other instances were in the group

uninstall.instance deletes entire Resource Group no matter what.

Help Needed (No action is triggered)

I have done all the steps and Getting no errors in Azure log streaming.
I have uploaded below test rule in azure functions.
When i close the child work item types eventually the parent has to get closed or show message. Nothing is happening. I am not getting any messages in Azure devops portal If Close parent or child. The states are also not getting transitioned automatically when I close child work items.
Please help

string message = "";
var parent = self.Parent;
if (parent != null)
{
var children = parent.Children;
if (children.All(c => c.State == "Closed"))
{
parent.State = "Closed";
message = "Parent was closed";
}
else
{
message = "Parent was not closed";
}
parent.Description = parent.Description + " aggregator was here.";
}
return message;

Feature : possibility to apply filter on webhook

Hello,

To limit traffic, it would be nice to be able to specify filters with the "map.rules" command.
For example :
dotnet .\aggregator-cli.dll map.rule --rule xxx --project "xxx" --event workitem.updated --resourceGroup xxx --instance xxx

--filtertype "Task" --filterfield "Microsoft.VSTS.Scheduling.OriginalEstimate"

And in Azure DevOps :

TFS Aggregator deployment failed on Devops

Please find attached images - we are unable to deploy TFS Aggregator extension on Devops. Please help us to troubleshoot it.

Error message is -
Deployment Error: The resource operation completed with terminal provisioning state 'Failed'!

Also, checked with Microsoft support (https://developercommunity.visualstudio.com/content/problem/656535/change-states-and-add-rules-for-inheritence-work-i.html?childToView=661455#comment-661455). They having same issue so referred to raise issue here,

Thank you!
Jolly Patel

The type 'TraceWriter' is defined in an assembly that is not referenced.

During rule execution.

2018-10-23T15:22:26.674 [Error] Function compilation error

Microsoft.CodeAnalysis.Scripting.CompilationErrorException : Script compilation failed.

at async Microsoft.Azure.WebJobs.Script.Description.DotNetFunctionInvoker.CreateFunctionTarget(CancellationToken cancellationToken) at C:\azure-webjobs-sdk-script\src\WebJobs.Script\Description\DotNet\DotNetFunctionInvoker.cs : 314

at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

at async Microsoft.Azure.WebJobs.Script.Description.FunctionLoader`1.GetFunctionTargetAsync[T](Int32 attemptCount) at C:\azure-webjobs-sdk-script\src\WebJobs.Script\Description\FunctionLoader.cs : 55

at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

at async Microsoft.Azure.WebJobs.Script.Description.DotNetFunctionInvoker.GetFunctionTargetAsync(Boolean isInvocation) at C:\azure-webjobs-sdk-script\src\WebJobs.Script\Description\DotNet\DotNetFunctionInvoker.cs : 183
2018-10-23T15:22:26.325 [Information] Executing 'Functions.testrule' (Reason='This function was programmatically called via the host APIs.', Id=ede53413-f440-4479-a8d6-fd90c1d1bfee)
2018-10-23T15:22:26.827 [Warning] run.csx(6,62): warning CS0618: 'TraceWriter' is obsolete: 'Will be removed in an upcoming version. Use Microsoft.Extensions.Logging.ILogger instead.'
2018-10-23T15:22:26.854 [Error] run.csx(8,23): error CS0012: The type 'TraceWriter' is defined in an assembly that is not referenced. You must add a reference to assembly 'Microsoft.Azure.WebJobs.Host, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null'.

Feature Request: Enable bypassRules param on batch api calls.

To eliminate some potential issues when using a rule with team projects using differing Processes the bypassRules param in the api calls can reduce the amount of issues in developing rules.

https://docs.microsoft.com/en-us/rest/api/azure/devops/wit/WorkItemBatchUpdate?view=azure-devops-rest-4.1&viewFallbackFrom=vsts-rest-4.1

Consider cancellation capability

Each asynchronous operation on the CLI and function stack should support cancellation.

• The entry points have to provide a CancellationToken
• Each synchronously called asynchronous method has to be awaited, therefore the public API changes

Advantages

• less resource allocations

Interesting PR commandlineparser/commandline#390

unmap.rule fails with "given key 'url' was not present in the dictionary

Hi,
I'm able to add rules to a TFS project but unable to delete them. Could you point me in the right direction for this issue ? I can't see where/why I add url to the args

This adds the rule successfully
dotnet c:\dev\tools\aggregator\aggregator-cli\aggregator-cli.dll map.rule --verbose --project "Test Failures Count Test" --event workitem.created --instance Aggregator --resourceGroup Azure-DevOps-Aggregator --rule update_effort_fields

But I'm unable to delete the rule
dotnet c:\dev\tools\aggregator\aggregator-cli\aggregator-cli.dll unmap.rule --verbose --project "Test Failures Count Test" --event workitem.created --instance Aggregator --resourceGroup Azure-DevOps-Aggregator --rule update_effort_fields

returns

[2019-04-23 15:44:31Z] Reading Azure DevOps project data...
[2019-04-23 15:44:31Z] Project Test Failures Count Test data read.
[2019-04-23 15:44:31Z] One or more errors occurred. (The given key 'url' was not present in the dictionary.)

Thanks in advance !
Dan

List.instances produces an error

When calling list.instances --resourceGroup XYZ --verbose an error is produced

One or more errors occurred. (Invalid version.
Parameter name: version)

Possible reason

Invoking xyz.scm.azurewebsites.net/api/vfs/site/wwwroot/aggregator-manifest.ini results in

version=0.4.5-RC

Manually invoking

	SemVersion.Parse(@"0.4.5-RC
").Dump();

in LINQPad results in an ArgumentException with the message

Invalid version.Parameter name: version

However manually invoking (pay attention to the missing last empty line)

	SemVersion.Parse(@"0.4.5-RC").Dump();

results in a correctly parsed version number

Analysis

The file src/aggregator-function/aggregator-manifest.ini only contains one line.
The file aggregator-manifest.ini in the deployment file FunctionRuntime.zip contains an empty last line

Possible Solutions

First

Update the PowerShell build task Set version in aggregator-manifest.ini to the last empty line is not produced

Second

Make the implementation of parsing the aggregator-manifest.ini file more robust

Update frequency to minimize

Hi,
I want to ask is there a way we can reduce the time it took to run the aggregator-cli script against an event, e.g. I have written a code to update the field (Notes) of a task to a standard value when any work item will be created. This is something like whenever a work item will be created its Notes field will have a default/pre-defined value.

I noticed that when I create work item it took between 10-25 seconds to update the field value.
I want to ask is there any configuration setting where we can reduce this time to something like 5 seconds?

Thanks,
Kamran

Log stream duplicates all messages

The log stream deployed by the CLI tool seems to show every message twice:

Any idea where I could look for a reason?