knom / vsts-regex-tasks

VSTS Regex Build/Release Tasks

RegexReplace Build/Release Task

A task for replacing REGULAR EXPRESSIONS in your files with variables and text in build and release pipelines.

Supported Platforms

• Azure DevOps - Pipelines
• Team Foundation Server 2015 Update 3 and higher (How to install extensions in TFS)
• Only Windows - sorry, this is due to the lack of Powershell support of the underlying task FX platform :-(
• Build Agents w. Powershell 3.0 or higher

Usage

1. Add the RegEx Find & Replace task to your build or release pipeline.

2. Configure the parameters

Parameters:

• Search Paths to Input Files:
  • The search patterns to the files that should used in Find & Replace
  • New in v3: Supports multiple input lines and minimatching
• Find RegEx:

  • The Regular Expression to FIND text

  • RegEx Groups (brackets) are used for the replace option

  • E.g. find all XML/HTML comments

        <!-- (.*) -->
    

• Replace With:

  • The text to replace with

  • $1 is the first group, $2 the second, ...

  • E.g. replace them with what was within the comment (comment it IN)

        $1
    

• New in v3: UseRAW (default true):
  • Whether to use RAW file read mode (might help with replace issues, such as newline)
• New in v3: UseUTF8 (default true):
  • Whether to read the file in UTF8 (otherwise in ASCII)

License

Now published under MIT License.

vsts-regex-tasks's Issues

Found 0 files to perform regEx replace

Hi There,

I am currently using this VSTS task to perform substitution for an angular 6 app, and having trouble getting it working.
I'm trying to perform substitution on the main..js file generated by the build. (where * is a random stamp with letter and number)
In the main..js i am trying to replace some properties e.g
someUrl:"http://localhost:4200"

here is what i've tried so far in the VSTS task
Search Pattern is **\*main.*.js
Find Regex: someUrl:"(http[a-zA-Z0-9.:/\-]+)"
Replace with someUrl:"$(someUrl)" where $(someUrl) is a variable defined in VSTS

When i look at the deployment logs i can see the following

Search Pattern is **\*main.*.js
Found 0 files
Replacing someUrl:"(http[a-zA-Z0-9.:/\-]+)" with someUrl:"https://mynewUrl.com"

No substitution has happened could tell me what i am doing wrong please?

Changes not being made

Hi, maybe it's a dumb question, but i'm trying for at least 3 hours to made it work...
I have a legacy code with many of this entries on Web.Config:

  <endpoint address="http://site1.com/123123/asdasd.svc"

I put this code on my Azure DevOps, and made a Pipeline Variabled called "$Hostname" with "site1.com".
So i changed all entries in my Web.Config with that:
<endpoint address="http://SITEVARIABLE/123123/asdasd.svc"

Now i just want to use the Regex Find and Replace to, when doing a Release, find every "SITEVARIABLE" and change with the Pipeline variable "$Hostname", so my code on server should be "the original" http://site1.com/123123/asdasd.svc.

I've tried a dozen of expressions and condictions, my last one is this:

Search Paths to Input Files:
*.config

Find RegEx
(SITEVARIABLE)

Replace With
$(Hostname)

No matter what i do, always the final web.config at my server is
<endpoint address="http://SITEVARIABLE/123123/asdasd.svc"

Could you help me to pointing what i'm doing wrong or what i didn't understood for the documentation?

Support \X token

Could you support \x

\X
Matches any valid Unicode sequence, including line breaks.

Sample

(item1\X*?.).(InnerItem : ").*(" )

item1{
InnerItem : "Target Item"
}
item2{
InnerItem : ""
}

All RegEx Find & Replace tasks in pipelines are suddenly failing for different reasons

Please see all the different error messages which we are getting suddenly

Using Variables and Backreferences in Replace

I'm trying to replace all strings in RC files that match 0[.,]0[.,]0 with major.minor.build or major,minor,build (depending on the first capture group's contents)

- task: RegexReplace@3
  displayName: 'Version Cpp'
  inputs:
    InputSearchPattern: '**\*.rc'
    FindRegex: '0([.,])0[.,]0'
    ReplaceRegex: '$(major)$1$(minor)$1$(build)'
    UseUTF8: true

When I run this, it properly populates the major, minor and build variables with their correct value, but the backreference to the first capture group is not being resolved - it is using a literal "$1"

How can I use both Build Variables AND backreferences in the same replace pattern?
Instead of using $1, I have also tried \1 and \1, all of which printed the literal string after evaluation.

Replace Connectionstring

Hello,

Please am trying to replace the connectionString value in the connection string below:
<connectionStrings> <add name="MS" connectionString="UseDevelopmentStorage=true;" /> </connectionStrings>

using this

but this is what i got

<connectionStrings> <add name="MS_AzureStoreConnectionString" connectionString="$(ReplacableToken_MS_AzureStoreConnectionString-Web.config Connection String_0)" /> </connectionStrings>

which is wrong, the correct value should be a normal azure storage connection string. Please how can i achieve this.

Thanks

Select a group within the Regex and replace it with a variable

I am using the following settings for the replace task.

Regex: "Host":"(\S*?)",
Replace with: $1(Host)

In the above regex, I want to select the first group. I have also added a variable as Url in the pipeline. But no difference can be seen. Am I doing this the wrong way?

Augment DEBUG logging and allow an empty string as a replacement value

The existing debug logging has been very helpful. But would it be possible to expand it to include information about the lines replaced (e.g. 'old line' -> 'new line')
If that is not possible, then specifying the number of tokens replaced in each file would be a good alternative.

Also, I would like to be able to enter an empty string as a regex replacement, but I do not see a way to do that currently.

Thank you in advance!

PowerShell version check doesn't fail the task

As I commented in #27 (comment), the PS4.0 check broke the task for us (we sincerely hope you will revert this check!). Unfortunately the task doesn't error, but just gives a message and signals DevOps that it executed successful. See also the provided screenshot.

I suspect that (besides the error you write) you need to exit the task with an error code or something equivalent.

how to replace with env variable, which its name is the captured group? (i.e. $3)

Hi,

My regex captures groups that indicate the name of the environment variable to replace with.
For example:
<---- (.+) ---->
I would like to replace with the environment variable named with what captured in group 1 (i.e. $1).

The path is not of a legal form

When I run this task, using a simple filter **\*.csproj, I get the following error. I've tried different variations of this, but all provide the same error.

Task : RegEx Find & Replace
Description : Find & Replace using Regular Expressions
Version : 2.0.1
Author : knom
Help : More Information

Search Pattern is **\*.csproj

Exception calling "GetDirectoryName" with "1" argument(s): "The path is not of a legal form."

Replacing an entire file does not function multiline

Using ^[\s\S]*$ to replace an entire file causes you to have a file with N lines of the replacement string, where N is the number of lines you have in your original config.

This extension doesn't work with multiple files

Lets say I have

$(agent.builddirectory)\s\ModuleMetaData.xml; $(agent.builddirectory)\s\ModuleMetaData.cs;
Then I need to replace the same thing in the multiple files. This extension should support this feature this is very important.

RegexReplace@3 turns the file into UTF-8 with BOM, how to export without BOM?

Hi there,

The RegexReplace@3 (YAML) extension turns the file into UTF-8 with BOM, how to export without BOM?

Thanks.

Pipeline step runs successfully but no change to code on branch

I am trying to use this to replace the path of some databricks notebook paths in a json files. The json files will have fields like this:
"notebookPath": "/Repos/<username>/<Repo>/notebooks/<path to notebook ...>",
I just need to change this to:

"notebookPath": "/notebooks/<path to notebook ...>",

I've used the following configuration:

`
steps:

• task: knom.regexreplace-task.regex-replace.RegexReplace@3
  displayName: 'RegEx Find & Replace: Replace Databricks Repo Paths in Pipelines'
  inputs:
  InputSearchPattern: 'pipeline/.json'
  FindRegex: '(/Repos/(.)/notebooks)/'
  ReplaceRegex: '$1(notebooks)'
  UseUTF8: false
  UseRAW: false
  `

But no change appears in the code on the branch.

we started facing issue with new version 3.3.0 (RegEx Find & Replace)

Hi,

we started facing issue with new version 3.3.0

Working fine with 3.2.1
Task : RegEx Find & Replace
Description : Find & Replace using Regular Expressions
Version : 3.2.1
Please check ASAP, our all builds started breaking.
Below error
##[error]Cannot convert value "System.String" to type "System.Boolean". Boolean parameters accept only Boolean values and numbers, such as $True, $False, 1 or 0.

Wrong error on empty "Replace With"

Hey,
Any other regex patterns are working but not this (I also tested other slashes):

Log:

Search Pattern is D:\a\3\a/**/Settings.psd1
##[error]The term 'replaceRegex' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

Search in: $(Build.ArtifactStagingDirectory)/**/Settings.psd1
Find RegEx: (\#\#\#User\#\#\#)
Replace with: $(UserDevelopment)

Regards,
TheAifam5

Mac Build Agent Support

Would it be possible to have a Mac Build Agent support so I can update and alter a file when running a Build on My Build Agent?

Can't perform simple substitution in files

I have an automated build set up for an Angular 5 project. The index.html and main.bundle.js files that are generated have values in them that need to be replaced with values stored in build variables.
In all cases, the values are quoted strings containing a URL or path.
I cannot seem to get the Regex Find & Replace task to work, and there's no useful info in the build logs.
For example, I might have a task to look in the dist directory (where the generated files are stored during build) for index.html, i.e.: projectName/dist/index.html.
I have tried various styles of regex, but none seem to make a difference. the value I want to match in this case is "./css/all.css".
I have tried setting the Replace With field to vaious different values, trying to point to a variable, e.g.: variableName, $variableName, $(variableName)...
What are the correct settings to make this work, and are they documented anywhere?

Linux build agent support

For the time being it looks like only Windows Build Agent servers are supported. Is there any plan for supporting Linux build agents?

2018-10-15T17:23:56.0658069Z ##[section]Starting: RegEx Find & Replace 2018-10-15T17:23:56.2662555Z ============================================================================== 2018-10-15T17:23:56.2944775Z Task : RegEx Find & Replace 2018-10-15T17:23:56.3220881Z Description : Find & Replace using Regular Expressions 2018-10-15T17:23:56.3409307Z Version : 3.2.1 2018-10-15T17:23:56.3597493Z Author : knom 2018-10-15T17:23:56.3942007Z Help : [More Information](https://github.com/knom/vsts-regex-tasks/) 2018-10-15T17:23:56.4214401Z ============================================================================== 2018-10-15T17:23:56.4865742Z ##[error]The current operating system is not capable of running this task. That typically means the task was written for Windows only. For example, written for Windows Desktop PowerShell. 2018-10-15T17:23:56.5317180Z ##[section]Finishing: RegEx Find & Replace

Getting "A parameter cannot be found that matches parameter name 'Raw'."

Having an issue (or misuse?) with this task. Here are the values I enter:

search paths to input file = ***.mysqlscript.sql
Find Regex = .\secondscript.sql
Replace With = .\thirdscript.sql

Use UTF = true
Use -Raw = True

running on self hosted agent (Powershell 5.1.14409.1018)

Task version = 3

And it gives me the error
"A parameter cannot be found that matches parameter name 'Raw'."

I can confirm the file was found the the Replacing step was shown.

Replace is case insenstitive

The RegEx replace performed is case insenstitive, since it uses the PowerShell Replace operator which by default is case insensitive (https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_comparison_operators?view=powershell-7.3#replacement-with-regular-expressions). This limits the usability of this task to situations where case is irrelevant only.

There is another operator "creplace" in Powershell which is cases sensitive. Would be good to at least have the option to use this to do case sensitive replacements.

All RegEx Find & Replace tasks in pipelines fail. The reason is that "A parameter cannot be found that matches parameter name 'NoNewLine'."

Image removed

UTF-8-BOM

After running this on my UTF-8 files, I get a UTF-8-BOM file which I cannot use in my pipeline, as the BOM causes the following processes to fail.

All of my files are prefixed with U+FEFF.

Encoding is always UTF-8

v3.2 of this task is always outputting with UTF-8 encoding. I have verified that I have the option ("Use UTF-8 Encoding") deselected in my build definition.