your-tools / ruplacer

Find and replace text in source files

Ruplacer

Find and replace text in source files:

$ ruplacer old new src/
Patching src/a_dir/sub/foo.txt
-- old is everywhere, old is old
++ new is everywhere, new is new

Patching src/top.txt
-- old is nice
++ new is nice

Would perform 2 replacements on 2 matching files.
Re-run ruplacer with --go to write these changes to disk

Note

This project was originally hosted on the TankerHQ GitHub organization, which was my employer from 2016 to 2021. They kindly agreed to give me back ownership of this project. Thanks!

Installing with cargo

Install rust and cargo, for example with rustup.

Then run:

cargo install ruplacer

Alternative installation methods

• Pre-compiled binaries for Linux, macOS, and Windows are available as assets of the latest release.

• ruplacer can also be installed from homebrew:

brew install TankerHQ/homebrew-repo/ruplacer

• ruplacer is also on the Arch Linux User Repository

Basic usage

ruplacer pattern replacement [path]

If the path is not given, it defaults to the current working directory.

Ruplacer will then walk through every file in <path> while honoring .gitignore files found on the way.

Binary files and text files containing non-UTF8 characters will be skipped. Then for every remaining file, it will read the contents, replace all lines matching the pattern by the replacement, and print the difference.

If you are OK with the replacements, re-run ruplacer with the --go option to actually write the changes to disk.

Regex

By default, pattern will be compiled into a Rust regex.

Note that it's slightly different from Perl-style regular expressions. Also, you must use $1, $2 to reference groups captured from pattern inside replacement.

For instance, this replaces 'last, first' by 'first last':

ruplacer '(\w+), (\w+)' '$2 $1'

(note the use of single quotes to avoid any processing by the shell)

If you don't want the pattern to be used as a regex, use the --no-regex command line flag.

This makes it possible to look for special characters without escaping them:

# This is a regex that matches the letter a
# or the letter o
$ ruplacer '(a|o)' u
- tata toto
+ tutu tutu
- (a|o)
+ (u|u)

# This is the literal string: '(a|o)'
$ ruplacer --no-regex '(a|o)' u
# or
$ ruplacer '\(a\|o|)' u
- (a|o)
+ u

Subvert mode

Ruplacer has a --subvert option which works across a variety of case styles (lower case, snake case, and so on):

$ ruplacer --subvert foo_bar spam_eggs
Patching src/foo.txt
-- foo_bar, FooBar, and FOO_BAR!
++ spam_eggs, SpamEggs, and SPAM_EGGS!

Filter files by type or glob patterns

Inspired by ripgrep, you can also select or ignore certain "file types" or glob patterns:

# Select only C++ files
$ ruplacer old new --type cpp
# Select only *.foo files
$ ruplacer old new --type *.foo
# Select only files that match foo*bar.c
$ ruplacer old new --type foo*bar.c

# Ignore all js files
$ ruplacer old new --type-not js
# Ignore all *.bar files
$ ruplacer old new --type-not *.bar
# Ignore all files that match foo*bar.c
$ ruplacer old new --type-not foo*bar.c

Each "file type" is just a list of glob pattern. For instance: the cpp file type matches *.C, *.H, *.cc, *.cpp and so on ...

You can see the whole list by using ruplacer --file-types.

ruplacer's Issues

Implement smarter replacement algorithm

For instance, when replacing foo_bar with spam_eggs, you may also want to replace FooBar with SpamEggs, FOO_BAR with SPAM_EGGS and so on ...

One lib we may use:
https://docs.rs/Inflector/0.11.3/inflector/

Tool seems to try to (re)write files that don't contain matches

$ echo "hello world" > a.txt
$ echo "foo bar baz" > b.txt
$ chmod 400 b.txt
$ ruplacer 'hello' 'goodbye'
./a.txt:1 - hello world
./a.txt:1 + goodbye world

Would perform 1 replacement on 1 matching file
Re-run ruplacer with --go to write these changes to the filesystem
$ ruplacer 'hello' 'goodbye' --go
Error: Could not write ./b.txt

Caused by:
    Permission denied (os error 13)

Use capturing group next to literal

Proposed tag: question.

I want to make all class {Name}Test to be internal class {Name}Test, so I try to run:

ruplacer 'class (\w+)Test' 'internal class $1Test'

But it doesn't work as expected. I avoided it with

ruplacer 'class (\w+)(Test)' 'internal class $1$2'

But how can solve my issue without adding second capturing group?

How to use Ruplacer in another function

I've tried to write an function:

fn run_ruplacer(path, foo, bar) {
   
}

but can not, is there any one can help me?

Idea: support replacing the content of stdin

For example:

cat myFile | ruplacer fooBar catBar

Not sure it makes sense, it is equivalent to using sed -e, but the syntax is easier with ruplacer.

Better diff algorithm

Right now if you replace foo_bar by spam_eggs, the underscore won't get colored when printing the diff.

I've tried and fail to fix the algorithm, so maybe we should use an other diff library instead.

Maybe this one: https://docs.rs/diff/0.1.11/diff/index.html ?

Or we port diff-so-fancy from Perl to Rust :P

Recursive abuse

Windows 7 x64, ruplacer 0.8.2.

The app behaves too self-willed, recursively diving into subfolders. How do I limit this behavior? For example, ripgrep implemented the flag --max-depth=N. It may be worth going backwards and not diving at all by default.

Bug with simple replace

I have some strange behavior with simple replace via windows git bash:

# ./ruplacer --no-regex --color never -t less  '~styles/' '@/styles/' ./src

Patching ./src\styles\widgets\calendar.less
-- @import '~styles/variables';
++ @import '@C:/Program Files/Git/styles/variables';

Version: ruplacer-v0.4.1-windows

Use `anyhow` for error handling

It's a good crate for our use case ;)

Error: nothing found to replace

Source: koi8-r.zip

$ ruplacer "<meta.*?charset=.*?>" $1 koi8-r.htm
Error: nothing found to replace 

but, hey, there is something to replace as follows

$ rg "<meta.*?charset=.*?>" koi8-r.htm
4:<META content="text/html; charset=koi8-r" http-equiv=Content-Type>

--rename feature for editing file names

Often in codebases, file names have to be changed alongside the text of the document. Being able to apply the same arguments and see the same style of preceding diff would be very helpful for unified editing of codebases.

Allow ruplacing in 'hidden' files

If a file starts with a dot but does not match any pattern in the .gitignore, it still gets ignored by ruplacer.

We should have a --no-hidden flag or something

Support multiline replace

I have the following in a lot of files:

          Log.send(
                new LogData(
                    this.baseData,
                    `${this.tryGetPingMatterLinkedToUser.name}.ts`,
                    userId,
                    Level.Warn,
                    `Dms client matter code: ${filedDocument.DmsClientMatterCode()} does not exist in Ping`
                )
            );

I want to replace all with logger.error(error).

I tried lots of different things, but I expected ruplacer 'Log\.send\((?s:.)*\);' 'logger.error(error)' -t ts to work, but it does not. Any thoughts?

--no-regex should be -F or --fixed-strings

Most other tools use these argument names. For example, grep:

-F, --fixed-strings       PATTERNS are strings

Or rg:

  -F, --fixed-strings                     
        Treat the pattern as a literal string instead of a regular expression. When
        this flag is used, special regular expression meta characters such as .(){}*+
        do not need to be escaped.

Regex error in documentation

In the documentation and help text, ruplacer '(\w)+, (\w)+' '$2 $1' should have the plus sign inside the parentheses: ruplacer '(\w+), (\w+)' '$2 $1'.

Question: Is there a tool as handy as Ruplacer, but for file renames?

I'd like to add an alias to my ~/.zshrc that calls Ruplacer and this other tool one after the other with the same parameters...

pattern/replacement starting with dashes

Hi,

Is there a way to have a pattern or replacement starting with dashes? Right now ruplacer think of them as invalid options.
Would it be possible to add "explicit" options? Something like ruplacer <-p|--pattern> --my-pattern-starting-with-double-dashes <-r|--replacement> -my-replacement-with-only-one-dash-at-the-beginning ?

Outdated version in brew

While trying to apply "--ignored" option I discovered that my version is 0.4.1 while newest is 0.6.4. I can't install if from assets (macOS warns me about it), but will try to install from source. It would be much easier to me if you will update your brew package version. Thanks.

Feature - file with list replacement operations

Create a file with replacement operations, regex or direct replacements, new line separated. Read this in as a batch operation.

Feature request: allow for replacement in normally ignored files

Currently ruplacer ignores things listed in .gitignore and also doesn't go into .git folders.
It should be nice to provide an option to the user to let ruplacer roam in the whole folder. Similar to the --hidden option of fd-find.

Ruplacing stdin

When using - as a path, ruplacer should read stdin and write to stdou.

This would mean we could also use ruplacer --subvert Foo Bar directly from text editors

Comparison to fastmod

Hey. Interesting project. I wondered: are you aware of https://github.com/facebookincubator/fastmod? It seems to have a similar goal as your project and is also written in Rust. I wondered if you wanted to include a comparison to it (somewhere).

Memory allocation failure when ruplacing in js bundle

bundle.zip

cargo run -- CONFIG_URLS foo-bar bundle

memory allocation of 38020685736 bytes failed

This looks fun ;)

Accept multiple paths on the command line

Currently ruplacer only accepts one path, unlike grep, sed, etc. I'd like to use it like

ruplacer foo bar folder1 folder2
ruplacer spam baz myproject/spam_*

Saving result in a diff file ?

Would it be possible to save the in a .diff file ? So it's possible to with git apply foo.diff to apply the change ?

If not, could you give a few pointers on which part of the code I should look at to add this feature ?

Tests suite fails on macOS

select 'ruplacement`by piping them though fzf or other selector

Hi !
First, thank for this amazing tool ! I use it almost on a daily basis :)
Sometime ruplacer over-deliver replacement candidate, and I wold like to select only a subset of them.
Would it be possible to pipe the candidate into a selector like fzf to achieve this purpose ?

If not, could you give me some pointer on how to achieve it ?

Feature request: filter files by regexp

Currently ruplacer provides a -t option to let work only on a file type, like c++, java, gradle ...
It would be nice if the user could provide a regexp to match against specific files and only check those.

Alternative installation options

Running cargo install is fine if you already have cargo, but people willing to try may not want to compile everything from sources

Some alternative installations options to implement:

Linux

• Publish .tar.gz containing Linux artifacts (x86_64)
• Publish .deb
• Publish AUR

macOS

• Publish .tar.gz containing macOS artifacts
• Homebrew recipe

Windows

• Publish zip containing Windows artifacts

Switch to GitHub Actions

So that we can have pre-compiled binaries for all 3 platforms

Possible wrong implementation of --ignored flag

In the help, it says that if --ignored flag is true, ruplacer will patch ignored files too. But current implementation and test case is the opposite. This has no effect because ignore::WalkBuilder::ignore is already set to true by default.

Error: nothing found to replace when no match found

In some cases, if no match is found, it's not a mistake.

For example: Read a folder with several files in it, then execute the ruplacer command for each file to replace, for example, "Hi" with "Bye" in each file read. If "Hi" is found, replace, otherwise skip (this should not be considered an error). If a match is found, OK, otherwise, SKIP. Wouldn't it be better to have a skip flag to prevent this behavior?

In my terminal, I get a lot of errors when there aren't any.

Other proposal: With ripgrep for example, it's possible to display only the modified items in output and not the entire file content + changes, especially for large files. This could be an idea for a new feature flag.

Replacement with empty string

Thanks for creating ruplacer! It's been great for renaming variables across source files.

Is there a way to replace matches with the empty string? For example,

ruplacer "string" ""

fails with

error: The following required arguments were not provided:
    <replacement>

I've been working around this by just adding an empty capture group () and then replacing with '$1'.

subvert does not replace all lower/upper case matches

Steps to reproduce:

echo 'stuffStuff stuffstuff STUFFSTUFF' > f
ruplacer stuffStuff fooFoo --subvert

Obtained result: only the first "stuffStuff" is replaced
Expected result: all three should be replaced

Argument `--type-list` requires other arguments

Expected --type-list to just list the types:

❯ ruplacer  --version
ruplacer 0.8.0
❯ ruplacer  --type-list
error: The following required arguments were not provided:
    <PATTERN>
    <REPLACEMENT>

USAGE:
    ruplacer --type-list <PATTERN> <REPLACEMENT>

For more information try --help

Workaround was to just spam some arguments:

❯ ruplacer  --type-list a b
Known file types:
agda: *.agda, *.lagda
aidl: *.aidl
...

How to install

This project looks awesome, but I can't figure out how to install it.

I tried typing ruplacer in my terminal, and it told me the command was not found
I tried typing ruplacer in google and it tooks me to a russian website, but I was affraid to command.

Please Help ;)

Feature request : add 'quiet' option

It will be nice to have -q/--quiet option to use in conjunction with --go. Or may be --qgo (for "quiet go").

Implement file renaming

ruplacer old new --go --rename-files

src/old.py -> src/new.py

Expose case-insensitive processing of `ignore` files?

The ignore crate provides an API for processing ignore-relevant files case-insensitively. Is this interesting to expose to end users?

Diff preview is suboptimal

When replacing trustchain by blockchain, we can get the following preview

--   trustchain_creation: 0,
++   blockchain_creation: 0,

Obtained result: trustchain_creation: is completely highlighted
Expected result: only the trustchain part should be highlighted, without the _creation:

Option for path sorting?

One thing that might be interesting for users: allow sorting of paths, which is useful for more deterministic output. walkdir allows results to be more strictly ordered, at the cost of (1) single-threading only, (2) some memory, and (3) latency for sorting. Suggested CLI: --sort-by=(file-name-natural|file-name|file-path|file-path-natural)

It's also pretty reasonable to not allow this -- if multiple paths were allowed, and it was guaranteed that paths would be visited in the order specified, then other shell commands can be used to provide an ordered list. This could also be complemented by being able to read a list of path arguments from a file.

replace patterns within files of a particular type

I'd like to replace patterns within files of a particular type, such as "--type rust". This feature is demonstrated within ripgrep.

`--ignored` seems not to work

Problem

I want to patch code of ignored files (dependencies of my python project). But there is no difference with or without --ignored - Error: nothing found to replace. At the same time ripgrep correctly finds provided pattern.

To reproduce

Libs used:

• python==3.10.2
• ripgrep==13.0.0 (or rg)
• ruplacer==0.6.4

It might be overcomplicated, but it is my real situation. Steps to reproduce:

mkdir reproduce
cd reproduce

git init
echo venv/ > .gitignore

python3 -m venv venv
source venv/bin/activate
pip install "Django==1.11.29"

# no any output since all files ignored by gitignore:
rg 'from collections import Iterable'
ruplacer 'from collections import Iterable' 'from collections.abc import Iterable'

# shows expected
rg 'from collections import Iterable' --no-ignore

# doesn't shows expected
ruplacer 'from collections import Iterable' 'from collections.abc import Iterable' --ignored

Shell completion

It would be great to have support for shell completion for bash, zsh, etc.

Support file renames

It doesn't seem like this is supported -- does this seem in scope?