The selfie from diffplug

JUnit 4

Apparently JUnit 4 is still more popular than JUnit 5 in Android land. The first project I want to use it on is also on JUnit 4 still.

The standard way to extend JUnit 4 is @RunWith, but it seems hard to have more than one, and you've gotta put it on every class. I think we can do everything we want with RunListener, but adding them to the test is not automatic by default.

There is a java-agent approach.

And Gradle seems to recommend JUnit vintage. That's probably the best way for us too, at least at first.

landing page CTA bug

open an incognito tab
go to https://selfie.dev/jvm/get-started
click "why" at the top
scroll down, it looks like this:

JS support for Kotest runner

JS seems like the easiest next step, but PRs for native platforms are welcome too!

in selfie-lib, move tests from jvmTest to commonTest one by one and implement the missing part
- most tests are already in commonTest, here is a suggested order for the 5 which are stuck in jvmTest
- LineReaderTest, SnapshotValueReaderTest, SnapshotReaderTest, SnapshotFileTest (sequentially in one PR)
- RecordCallTest in its own PR, perhaps needs to stay platform-specific
in selfie-runner-kotest search for TODO: https://github.com/diffplug/selfie/issues/186
move HarnessVerifyTest out of jvmTest and into commonTest

Favicon

At smallest size it might have to be only this

But if possible it would be better to be this

You can erase the "ear" and slide the face closer to the phone, at small pixel sizes I don't think anyone will notice. Might be worth trying ChatGPT-V for this...

Add quickstart section on overwrite all

(nit) add better bottom to the quickstart page

Our quickstart has a good section on CI. That should segue into a section on "overwrite all". Doing that is different in Gradle vs Maven.

Gradle

test { 
  systemProperties project.properties.subMap(["selfie"]) // works for groovy, confirm for Kotlin too
}

Then call ./gradlew test -Pselfie=overwrite.

Maven

Unknown, TBD.

Inline snapshot roadmap

The bones of inline snapshots landed in:

#23

Things which we'll need to implement

Document `memoize` (maybe `lazySelfie`?)

Use https://github.com/aallam/openai-kotlin as an example.

Might want to change our navbar to be:

why
get started
memoize
facets

or

why
get started
lazy
facets

Might also want to change the landing page, maybe: literal, lensable, lazy, and like a filesystem.

The function name lazy is already taken by kotlin-stdlib. Renaming to lazySelfie might be a better fit.

EDIT: landed on cacheSelfie, because memoizing random data is bad, and selfie is good for that.

Add a mode which preserves carriage returns

Here are two important properties of the .ss snapshot files:

exactly character-for-character accurate, no slop around leading or trailing whitespace, no forbidden characters
what you see in the snapshot file is exactly what you get, except for the following escaping rules which preserve the above
- the following characters are escaped: 𐝃 -> 𐝃𐝃, 𐝁 -> 𐝃𐝁 (they are from an untranslated dead language)
- if the first character on a line within a snapshot is ╔ then it is replaced with 𐝁 (this preserves ASCII art within snapshots)

This combination of character-accurate + WYSIWYG breaks down in only one place - line endings. You can't see them, and git's complex and poorly understood line-ending-mutation rules mean that most teams can't reliably do source control that differentiates between \n and \r\n.

Rather than randomly punch users in the face with this triviality, we do the following:

Internally, every text-based snapshot in spotless-snapshot has a .replace("\r", ""), so snapshots will never fail because of line-ending differences
New .ss files are always written using \n line endings
If an .ss file is loaded from disk with \r\n, it is converted and parsed using \n, but will be written back to disk as \r\n

However, this means that the snapshots are not exactly character-for-character accurate because they do not preserve \r. For users that want to preserve \r, we could add a mode which preserves the \r character if the user uses the .ss.asar format

#1

Adding support for storing \r within .ss files might be possible (maybe encode them as 𐝃r?), but doesn't seem like a good idea.

Emoji snapshot methods

We will never ship this by default, but it might make sense to do something like what blowdryer did for 干

com.diffplug.selfie:selfie-kotlin-emoji
- expectSelfie -> 🔎
- this would only work in kotlin, and it would have to be bracketed with double-backticks, but it stands out visually
- Unicode has both 🔎 and 🔍, doesn't really matter which, but we should not do both and :mag_right: works in GitHub but :mag_left: doesn't so that's the tiebreaker
- another option is expectSelfie -> 📷
com.diffplug.selfie:selfie-ku
- expectSelfie -> 𐌒
- this is using the old italic letter ku, which would work in java and kotlin

The code would look something like this

object 𐌒 {
  @JvmStatic fun 𐌒(actual: String) = Selfie.expectSelfie(actual)
  @JvmStatic fun 𐌒(actual: Int) = Selfie.expectSelfie(actual)
  @JvmStatic fun 𐌒(actual: Boolean) = Selfie.expectSelfie(actual)
}

It's cute, but it's very easy for people to do themselves in their own projects, and it's probably a bad idea so we shouldn't encourage it by publishing artifacts for it.

The 🔎 has a tiny sliver of functionality, the color does standout just a bit, seems appropriate.

Implement assertion library

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

Update dependency @types/node to v20.11.27
Update dependency framer-motion to v11.0.13
Update dependency io.jooby:jooby to v3.0.9
Update dependency io.jooby:jooby-netty to v3.0.9
Update dependency io.jooby:jooby-test to v3.0.9
Update dependency pyright to v1.1.354
Update ver_KOTEST to v5.8.1 (io.kotest:kotest-assertions-shared, io.kotest:kotest-framework-engine, io.kotest:kotest-assertions-core, io.kotest:kotest-runner-junit5)
Update ver_OKIO to v3.9.0 (com.squareup.okio:okio-nodefilesystem, com.squareup.okio:okio)

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

Update plugin dev.adamko.dokkatoo-html to v2.2.0

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

github-actions

.github/workflows/gradle-wrapper-validation.yml

actions/checkout v4

gradle/wrapper-validation-action v2

.github/workflows/jvm-ci.yml

actions/checkout v4

actions/setup-java v4

gradle/actions v3

mikepenz/action-junit-report v4

.github/workflows/jvm-deploy.yml

actions/checkout v4

actions/setup-java v4

gradle/actions v3

.github/workflows/jvm-publish-kdoc.yml

actions/checkout v4

actions/setup-java v4

gradle/actions v3

cloudflare/pages-action v1

.github/workflows/python-ci.yml

actions/checkout v4

actions/setup-python v5

gradle

jvm/gradle.properties

org.junit.jupiter:junit-jupiter 5.10.2

org.jetbrains.kotlinx:kotlinx-serialization-json 1.6.3

io.kotest:kotest-runner-junit5 5.4.0

io.kotest:kotest-assertions-core 5.4.0

io.kotest:kotest-framework-engine 5.4.0

com.squareup.okio:okio 3.8.0

org.jetbrains.kotlin:kotlin-test-junit5 1.9.23

org.junit-pioneer:junit-pioneer 2.2.0

io.kotest:kotest-assertions-shared 5.4.0

com.squareup.okio:okio-nodefilesystem 3.8.0

org.junit.vintage:junit-vintage-engine 5.10.2

jvm/settings.gradle

com.diffplug.blowdryerSetup 1.7.1

com.diffplug.spotless 6.25.0

com.diffplug.spotless-changelog 3.0.2

com.gradle.plugin-publish 1.2.1

dev.equo.ide 1.7.6

io.github.gradle-nexus.publish-plugin 2.0.0-rc-2

org.jetbrains.dokka 1.9.20

org.jetbrains.kotlin.jvm 1.9.23

org.jetbrains.kotlin.plugin.serialization 1.9.23

org.jetbrains.kotlin.multiplatform 1.9.23

dev.adamko.dokkatoo-html 2.0.0

jvm/build.gradle

jvm/example-junit5/build.gradle

io.jooby:jooby 3.0.8

io.jooby:jooby-netty 3.0.8

jakarta.mail:jakarta.mail-api 2.1.3

io.jooby:jooby-test 3.0.8

io.rest-assured:rest-assured 5.4.0

org.assertj:assertj-core 3.25.3

org.jsoup:jsoup 1.17.2

com.vladsch.flexmark:flexmark-html2md-converter 0.64.8

jvm/example-kotest/build.gradle.kts

com.aallam.openai:openai-client-bom 3.7.0

jvm/gradle/spotless.gradle

jvm/gradle/dokka/dokkatoo.gradle

jvm/selfie-lib/build.gradle

jvm/selfie-runner-junit5/build.gradle

jvm/selfie-runner-kotest/build.gradle

jvm/undertest-junit-vintage/build.gradle

junit:junit 4.13.2

jvm/undertest-junit5/build.gradle

org.jetbrains.kotlinx:kotlinx-coroutines-core 1.8.0

jvm/undertest-junit5-kotest/build.gradle

jvm/undertest-junit5-kotest/harness/gradle.properties

jvm/undertest-junit5-kotest/harness/settings.gradle

com.diffplug.blowdryerSetup 1.7.1

com.diffplug.spotless 6.25.0

com.diffplug.spotless-changelog 3.0.2

com.gradle.plugin-publish 1.2.1

dev.equo.ide 1.7.6

io.github.gradle-nexus.publish-plugin 2.0.0-rc-2

org.jetbrains.dokka 1.9.20

org.jetbrains.kotlin.jvm 1.9.23

org.jetbrains.kotlin.plugin.serialization 1.9.23

org.jetbrains.kotlin.multiplatform 1.9.23

dev.adamko.dokkatoo-html 2.0.0

jvm/undertest-junit5-kotest/harness/build.gradle

jvm/undertest-junit5-kotest/harness/gradle/spotless.gradle

jvm/undertest-kotest/build.gradle

gradle-wrapper

jvm/gradle/wrapper/gradle-wrapper.properties

gradle 8.6

npm

selfie.dev/package.json

@headlessui/react ^1.7.13

@mdx-js/loader ^3.0.0

@mdx-js/react ^3.0.0

@next/mdx ^14.0.0

@sindresorhus/slugify ^2.1.1

@tailwindcss/typography ^0.5.8

acorn ^8.8.1

autoprefixer ^10.4.7

clsx ^2.0.0

flexsearch ^0.7.31

framer-motion 11.0.8

mdast-util-to-string ^4.0.0

mdx-annotations ^0.1.1

next ^14.0.0

react 18.2.0

react-dom 18.2.0

react-highlight-words ^0.20.0

recma-nextjs-static-props ^2.0.0

rehype-mdx-title ^3.0.0

remark ^15.0.1

remark-gfm ^4.0.0

shiki ^0.14.0

tailwindcss ^3.3.0

unist-util-visit ^5.0.0

zustand ^4.3.2

@types/node 20.11.25

eslint 8.57.0

eslint-config-next 14.1.3

prettier ^3.0.0

prettier-plugin-tailwindcss ^0.5.0

typescript 5.3.3

nvm

selfie.dev/.nvmrc

node 20.11.1

pep621

python/selfie-lib/pyproject.toml

poetry

python/selfie-lib/pyproject.toml

python ^3.12

ruff ^0.3.0

pyright ^1.1.350

pytest ^8.0.0

pyenv

python/.python-version

python 3.12

Check this box to trigger a request for Renovate to run again on this repository

Add support for multiline string literals in Groovy

see #90 for a tour of the relevant spots

Binary-optimized snapshots

The .ss file extension we are using right now is optimized for text. It would be good to have a snapshot extension optimized for binary. I think asar is perfect for that, using .ss.asar.

Add JPMS module-info.java

I'm not sure that anyone will ever need this. But I mistakenly thought that we did to reproduce a user bug, so I started experimenting in the feat/module-info branch (see diff).

The hiccups I ran into were:

the module-info.java has to be in a java folder, not a kotlin folder
stuff that is in the java folder can't reference stuff in the kotlin folder
we needed fake package-private NeededForModuleInfo.java files in the java folder to avoid compilation errors in the module-info.java

If you need selfie to support module-info.java, I'm happy to merge a PR that does it, and you can use this as a starting point.

the hiccups above are fine with me. If you can fix the workarounds great, but I don't mind shipping them
please merge in from main first, maybe the latest version of the Kotlin plugin will have made it easier by then
I would like the metadata (naming and whatsuch) to be canonical, and I don't have enough experience with JPMS to know what it should be. Hopefully if you need JPMS, you've also got enough experience that you can set this metadata better than I did.

Improve mascot position/animation

Desired behavior:

Mascot is behaving perfectly today at screen widths smaller than 1300px. Keep this behavior.
At 1300px and above, the mascot should remain fixed relative to the 1300px wide content column. It should not be fixed to the left edge of the viewport.
At 1300px and above, the mascot scroll animation should be pretty much identical to the scroll animation at 1299px.

Current behavior:

To get the mascot fixed relative to the content column, I changed its position from fixed to absolute at the 1300px breakpoint. However, this breaks the scroll animation. I tried messing with the calculations in the animation config with absolute positioning but I couldn't come close to getting it to work.

I think the solution is to keep the mascot with fixed positioning to restore scroll animation behavior and to use translateX and left in some combination to keep it inside that 1300px content column.

Migrate `PerCharacterEscaper` from JVM-only to multiplatform

Inside the snapshot-lib project you will see these folders:

commonMain - compiles for all platforms
jvmMain - compiles only on the JVM
jsMain - compiles only to JS
commonTest - these tests run on all platforms
jvmTest - these tests only run on JVM
etc.

Good example

We want as much code as possible to live in common. A good example of multiplatform is this:

a simple internal expect fun to wrap a platform-specific API
all the logic of ArrayMap lives in common

https://github.com/diffplug/spotless-snapshot/blob/f63192a84390901a3d3543066d095ea23bf81d21/snapshot-lib/src/commonMain/kotlin/com/diffplug/snapshot/ArrayMap.kt#L22-L115

Then we implement the "expect" for both jvm and js with "actual"

Work to be done.

In this case, we have JVM-only code. It should be pretty easy to move it into common, but there are a few tricky parts, which I don't want to deal with right now. So I move only the public API into common.

https://github.com/diffplug/spotless-snapshot/blob/f63192a84390901a3d3543066d095ea23bf81d21/snapshot-lib/src/commonMain/kotlin/com/diffplug/snapshot/PerCharacterEscaper.kt#L18-L21

Leave the implementation as jvm only, and the js just throws TODO exceptions for now

To implement this I would

move PerCharacterEscaperTest out of jvmTest and into commonTest
- gradlew jvmTest will still be passing, but gradlew jsTest will now be failing
remove the expect actual stuff and just put the whole JVM implementation into common
fix compile errors with as little platform-specific code as possible

Add github-style anchor links to the docs

get-started and advanced would benefit from them greatly
jvm landing could probably use them too

/r/java launch

post to

https://www.reddit.com/r/java/ for who we can help now
https://discuss.kotlinlang.org/, https://www.reddit.com/r/Kotlin/ for who can help us to multiplatform
https://www.reddit.com/r/javascript, https://www.reddit.com/r/programming, HackerNews after we are multiplatform

Let's see what GitHub does

github does this
- and this
interesting

Javascript roadmap

make our kotest runner support js (#186)
create selfie-runner-mocha for Node.js testing
create selfie-runner-karma for in-browser testing
- piping the filesystem to/from the browser is tricky, but karma-snapshot figured it out
- this is why the FS is provided by SnapshotStorage

Add `infix fun String.selfieIs` (similar to Kotest `shouldBe`)

Kotest has a very nice shouldBe method:

someFunction() shouldBe "some value"

It would be handy if Selfie had an infix version:

someFunction() selfieIs null // equivalent to `toBe_TODO`, run and it becomes
someFunction() selfieIs "some value"

And all the //selfieonce, //SELFIEWRITE would work too if implemented roughly like so:

// in com.diffplug.selfie
infix fun String.selfieIs(expected: String?) : Unit {
  if (expected == null) {
    Selfie.expectSelfie(this).toBe_TODO()
  } else {
    Selfie.expectSelfie(this).toBe(expected)
  }
}

// in com.diffplug.selfie.coroutines
suspend infix fun String.selfieIs(expected: String?) : Unit {
  if (expected == null) {
    Selfie.expectSelfie(this).toBe_TODO()
  } else {
    Selfie.expectSelfie(this).toBe(expected)
  }
}

The main challenge will be adapting this code to parse infix multiline literals

selfie/jvm/selfie-lib/src/commonMain/kotlin/com/diffplug/selfie/guts/SourceFile.kt

Lines 105 to 111 in 952a4eb

    
           fun parseToBe_TODO(lineOneIndexed: Int): ToBeLiteral { 
        
             return parseToBeLike(".toBe_TODO(", lineOneIndexed) 
        
           } 
        
           fun parseToBe(lineOneIndexed: Int): ToBeLiteral { 
        
             return parseToBeLike(".toBe(", lineOneIndexed) 
        
           } 
        
           private fun parseToBeLike(prefix: String, lineOneIndexed: Int): ToBeLiteral {

Python / .NET / Go / other langauges

Since Selfie is written in multiplatform Kotlin, it could be compiled to a native module and used in other ecosystems. But it's only ~2k lines, so it's probably easier to just rewrite in the host language.

#170
#84
hopefully other languages to come!

Create a separate Gradle build for launching the undertest `./gradlew build` fails (needs `:selfie-runner-junit5:testKotest` first)

The undertest situation is weird, we're kind of abusing Gradle a bit.

If you clone this project and run ./gradlew build, it will fail.

If you run ./gradlew testKotest and then ./gradlew build, it will pass. That is what we do on CI.

This is bad. It happens because of a "bug" in Gradle but it's really our fault:

you call ./gradlew build which runs tests
while those tests are running, some of them call ./gradlew :undertest-blah:test, on the very same project you are executing right now!
no wonder there's a bug

selfie/jvm/selfie-runner-junit5/src/test/kotlin/com/diffplug/selfie/junit5/HarnessJUnit.kt

Lines 211 to 215 in b8529be

    
           fun gradlew(task: String, vararg args: String): AssertionFailedError? { 
        
             return GradleConnector.newConnector() 
        
                 .forProjectDirectory(subprojectFolder.parent!!.toFile()) 
        
                 .connect() 
        
                 .use { connection ->

The "right" thing would be to create some kind of test harness project...

Individual (package private) test method execution stomps snapshot state from other methods

I’m finding that if I:

Have a FooTests.java with multiple selfie-taking test methods
Successfully generate FooTests.ss, with labelled sub-snapshots
Execute only (e.g.) fooTest1() via IntelliJ (using Gradle test runner)

…that this results in FooTests.ss being updated to contain only the snapshot state for the individually-executed method, thereafter causing fooTest2(), and so on to fail.

Might this be another situation where GC is too aggressive on account of package-private methods?

Side note: Rewriting the .ss file at all when there are no control comments or _TODO()s in play feels a little confounding in general — might it be worth gating all GC/snapshot writes behind those, or else perhaps the invocation of a clean (or other dedicated) task?

Type error in _app.tsx

Just noticed this after merging #24. Possibly related to the Code component, but not sure. See components prop of MDXProvider in _app.tsx

selfie/docs/src/pages/_app.tsx

Line 16 in 5a50cff

Type 'typeof import("/Users/toddriley/projects/selfie/docs/src/components/mdx")' is not assignable to type 'MDXComponents | MergeComponents | null | undefined'.
  Type 'typeof import("/Users/toddriley/projects/selfie/docs/src/components/mdx")' is not assignable to type 'MDXComponents'.
    Type 'typeof import("/Users/toddriley/projects/selfie/docs/src/components/mdx")' is not assignable to type '{ symbol?: Component<SVGProps<SVGSymbolElement>> | undefined; object?: Component<DetailedHTMLProps<ObjectHTMLAttributes<HTMLObjectElement>, HTMLObjectElement>> | undefined; ... 174 more ...; view?: Component<...> | undefined; }'.
      Types of property 'code' are incompatible.
        Type '({ children, ...props }: { [x: string]: any; children: any; }) => Element' is not assignable to type 'Component<DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>> | undefined'.
          Type '({ children, ...props }: { [x: string]: any; children: any; }) => Element' is not assignable to type '(props: DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>) => ReactNode'.
            Types of parameters '__0' and 'props' are incompatible.
              Type 'DetailedHTMLProps<HTMLAttributes<HTMLElement>, HTMLElement>' is not assignable to type '{ [x: string]: any; children: any; }'.
                Property 'children' is optional in type 'ClassAttributes<HTMLElement> & HTMLAttributes<HTMLElement>' but required in type '{ [x: string]: any; children: any; }'.ts(2322)
index.d.ts(50, 5): The expected type comes from property 'components' which is declared here on type 'IntrinsicAttributes & Props'

We could probably also benefit from a tsc CI check.

Add support for Kotest

If you are only using Kotest with JUnit5, then Selfie will work perfect out of the box with selfie-runner-junit5.

If you want, you can instead use selfie-runner-kotest. This has the slight disadvantage that you will have to register selfie with Kotest, while selfie-runner-junit5 requires no registration. But it has the advantage that it works with Kotlin Multiplatform, so you can take your selfies anywhere that Kotlin and Selfie can run.

Other differences between selfie-runner-kotest and selfie-runner-junit5

system property selfie.settings -> env SELFIE_SETTINGS

This issue is tracking the completion of the JVM version of selfie-runner-kotest. For JS, see #186. Other platforms are welcome as well.

create selfie-runner-kotest and undertest-kotest #185
copy selfie-runner-junit's Harness class into a HarnessKotest #185
copy FSJava and name it FSOkio, implement with okio #188
copy SnapshotFileLayoutJUnit and name it SnapshotFileLayoutKotest #191
now the tricky part, implement SnapshotStorage, which should work in combination with Kotest's prepareSpec/finalizeSpec/beforeTest/afterTest hooks #201
the hardest part is garbage-collecting stale snapshots, leave this out at first #216

`toBeFile` in the memos is great, exposes a weakness elsewhere...

In our memoizing API, we don't have any concept of facets, so it's a bit simpler.

StringMemo<T>
- toMatchDisk(sub: String = "") : T
- toBe(expected: String): T
BinaryMemo<T>
- toMatchDisk(sub: String = ""): T
- toBeFile(subpath: String): T

I really like this structure. You always have the toMatchDisk() which puts it into a selfie-managed file that doesn't care whether it's a string or binary. For strings, you can leave selfie's file management behind and store it as an inline String with toBe. And for binary, it makes sense that you might want to put a certain extension on it and view it in a normal file explorer, in which case you say toBeFile and you can specify the correct file extension and all that.

I think toBe for string data is perfect, and toBeFile is the perfect analogy for the binary case. The name might not be perfect.

Good parts
- toBe is perfect, toBeFile piggy backs on that well
- the File part makes clear that it's not inline, it's at this file
Bad parts
- toBeFile sounds a little bit like toMatchDisk, although it's quite different

Meh. Good enough.

Facets

What we have now in 1.x: assertion selfies can have "facets", which makes them a bit more complex than memo selfies.

fun expectSelfie(actual: String) = expectSelfie(Snapshot.of(actual))
fun expectSelfie(actual: ByteArray) = expectSelfie(Snapshot.of(actual))
fun expectSelfie(actual: Snapshot) = DiskSelfie(actual, deferredDiskStorage)

class DiskSelfie : LiteralStringSelfie {
  fun toMatchDisk(sub: String = ""): DiskSelfie
}
class LiteralStringSelfie {
  fun toBe(expected: String) : LiteralStringSelfie
  fun facet(facet: String) = LiteralStringSelfie(actual, listOf(facet))
}

The good thing about this is that you can do fluent chaining

expectSelfie(complexHtml).toMatchDisk()
  .facet("statusCode").toBe("200")
  .facet("md").toBe_TODO()

The bad thing is that we never really know what type we're going to get.

We could add facetBinary("png").toBeFile("screenshot.png"). The problem is that ideally

expectSelfie(actual: ByteArray) should return something which has toMatchDisk and toBeFile
once you call facet, you should lose the ability to say toMatchDisk
- the point of toMatchDisk is to dump it and only look at it again if it changes
- the point of facets is to pull important parts of those dumps out and do inline assertions with toBe

Add support for multiline string literals in Scala

see #90 for a tour of the relevant spots

Base64 encoding

We allow users to make binary SnapshotValue instances, but we don't support actually encoding them to disk or inline snapshots.

Fixing this is pretty straightforward, I think the main undecided thing is where to put newlines. 80 column width or somesuch is fine, but we should also add a newline whenever there's a 0x0000 or something like that, to reduce diff noise when a changed binary file hasn't completely changed. Maybe look at the zip spec, and see how we could break at the start of each entry, something like that...

Gradle up-to-dateness

It's important that snapshot files are included in up-to-date checks. We should get to the bottom of when they are / aren't included, and figure out how best to fix it.

Python implementation (in progress)

Ruff seems like a no-brainer
for static typing, seems like MyPy --strict vs pylyzer
- the other static checkers seem to have affordances for transitioning untyped code, but we are greenfield so I don't think they make much sense
- reddit advice is MyPy --strict, I'll go with that

Glossary

Subject - a single value (string or binary) which best records a snapshotted entity
Facet - a single named value (string or binary) which records a single aspect or point of view of a snapshotted entity
Snapshot - a combination of a subject and zero or more facets which captures all relevant data of an entity whose behavior we wish to record and specify
Camera - a function that transforms a specific type of entity into a snapshot
Lens - a function which transforms one snapshot into a new snapshot, usually by adding new facets that emphasize a certain point of view of the subject, but possibly also by cleaning the existing subject and facets
Equality assertion - a success condition which requires a value under to test to exactly match an expected value
Inline snapshot - an equality assertion where the expected value can be written into the sourcecode by automated tooling
Inline literal snapshot - an inline snapshot where the expected value must always be a source code literal, never a compound expression
- expect(2 + 2).toMatchInlineSnapshot(4) <-- inline literal snapshot
- expect(2 + 2).toMatchInlineSnapshot(2 * 2) <-- inline ~~literal~~ snapshot, because it's not literal, when tooling modifies the value inside (), it can't do a self-error-check to make sure it has correctly parsed the same value which got passed at runtime
- Conversationally people will say "inline snapshot", "literal snapshot", "inline literal snapshot", etc. They mostly mean the same thing, but in selfie we will try to always say "inline literal".

If you think any of these terms are unclear, feel free to discuss below. If there are any other concepts in the code which should be defined explicitly, feel free to discuss below. The final result of all discussions is incorporated into the list above via editing, after which the discussions may be deleted. This glossary is based on an idea from Rich Hickey's Design in Practice talk.

Improve read/write control

Google sheet, background on how decision matrix works.

EDIT: final result, the bold got implemented, see quickstart for enduser docs.

Add memoizing API

It would be useful to be able to do something like

val result = memoize { expensiveNonDeterministicApiCall() }.toBe("result from first execution")
val result = memoize { expensiveNonDeterministicApiCall() }.toMatchDisk()
buildOtherStuffOnMemoizedResult(result)

A prototype of how we could implement something like this with our existing primitives is here.

A trick is that a lot of what we want to cache is likely to be domain objects which support kotlin serialization. An API along these lines would be great (along with suspend versions with suspend arguments).

memoize(generator: () -> String) : StringMemo<String>
memoize(parser: Roundtrip<T, String>, generator: () -> String) : StringMemo<String>
memoizeAsJson<T>(generator: () -> T) : StringMemo<T> // for T with @Serializable

Policy for when to read vs write snapshots

There is no distinction between writing disk snapshots vs literate snapshots. It's the same flag/mode. The pioneer here, Jest, works like this: --update-snapshot or -u

For Selfie, I think the default should be:

if environment variable CI is set to true -> READ
else -> WRITE
if environment variable or system property selfie is set to either read or write, then that overrides the behavior above

Escaping leading space in multiline string literals is annoying.

In a multiline string literal, we can't trust that leading spaces or tabs will be preserved, so we escape the first one on each line.

This can be annoying, it would be nice if we supported a setting like:

interface SelfieSettingsAPI {
  val escapeLeadingWhitespace: EscapeLeadingWhitespace? = null
}
enum EscapeLeadingWhitespace {
  ALWAYS,
  NEVER,
  ONLY_ON_SPACE,
  ONLY_ON_TAB,
}

And for the default value of null, we:

look at all the leading whitespace in the file as it exists right now
is all of it space? EscapeLeadingWhitespace.ONLY_ON_TAB
is all of it tab? EscapeLeadingWhitespace.ONLY_ON_SPACE
a mixture / not enough to tell? EscapeLeadingWhitespace.ALWAYS

I bet if we implement this default no one will even ask for the ability to set a specific setting.

should not garbage collect snapshots when all tests are package-private

Hello! Excited to start using this project.

I have a test for some generated HTML that is currently using a hard-coded fixture (barely tolerable with text blocks), and converting it with Selfie.expectSelfie(output).toMatchDisk_TODO() was a breeze.

I am having an issue with snapshot persistence however, wherein a subsequent Gradle invocation removes the artifact, and thereafter the test fails unless I add //SELFIEWRITE (or revert to the _TODO() invocation).

I'll try to add more details later (environment, etc.), but for now, the useful error breadcrumb I see is:

java.lang.AssertionError: Selfie wrote a snapshot and then marked it stale for deletion it in the same run: <path snipped>
Selfie will delete this snapshot on the next run, which is bad! Why is Selfie marking this snapshot as stale?
    at com.diffplug.selfie.junit5.Progress.finishedAllTests(SelfieTestExecutionListener.kt:316)

Let me know if there's anything else besides environment info I can add later that might help track this down. I can take a swing at an MVCE if necessary too. Thanks!

Implement `SnapshotReader` and `SnapshotFile`

The key thing to note here is that SnapshotValueReader reads SnapshotValue

https://github.com/diffplug/spotless-snapshot/blob/f63192a84390901a3d3543066d095ea23bf81d21/snapshot-lib/src/commonMain/kotlin/com/diffplug/snapshot/SpotlessFile.kt#L18-L21

But the functions we're implementing here operate on a full Snapshot.
Most Snapshot will just be a single SnaphotValue.of("some string"). But sometimes there will be more. We can take for granted that a snapshot file will be sorted, so the example below will parse into 2 Snapshot (out of 4 SnapshotValue):

╔═ test ═╗
╔═ test[lens 1] ═╗
╔═ test[lens 2] ═╗
╔═ otherTest ═╗

And the example below can fail with an error. Probably the error should be something like "test[lens 2] (L:37)" was out of order.

╔═ test ═╗
╔═ test[lens 1] ═╗
╔═ otherTest ═╗
╔═ test[lens 2] ═╗

Code block styling

Make code blocks responsive. Ideally the height of the text and of the gap between lines would be the same in the code examples as in the body text above it.

See #30 (comment)

`cacheSelfie().xxx_TODO()` should cascade through later `cacheSelfie` calls

A classic problem in Jupyter / IPython workflows is:

you run the whole notebook
you make some changes and run just the first cell
you look at the last cell which has cached results from the initial run, but has not updated since the changes in the first cell, and you are confused

If you use //selfieonce or //SELFIEWRITE, these sorts of problems don't happen. But with _TODO they can. We should probably have something like this in our settings:

enum CacheTodoCascade {
  NONE, // This is our current behavior
  WITHIN_TEST, // Once a _TODO is encountered in a test,
               // it will rewrite everything after that in the test.
               // This should probably be the default.
  WITHIN_TEST_CLASS, // Rewrite everything in all the rest of the tests in that class
                     // This introduces coupling between tests, which is ill-defined if they are
                     // being executed in parallel. Might be not be a good idea...
}

This is quite hazardous with cacheSelfie. With expectSelfie it's not hazardous but can be annoying. I'm inclined for this functionality to be limited only to CacheTodoCascade, but it's worth considering just TodoCascade and use it for both expectSelfie and cacheSelfie calls.

`toBeFile` has bad error message when the file doesn't exist

We also don't catch duplicate writes with the same vigor as we do for toBe and toMatchDisk. Probably need some changes around here

selfie/jvm/selfie-lib/src/commonMain/kotlin/com/diffplug/selfie/guts/WriteTracker.kt

Lines 99 to 105 in 0c7e40a

    
           class DiskWriteTracker : WriteTracker<String, Snapshot>() { 
        
             fun record(key: String, snapshot: Snapshot, call: CallStack, layout: SnapshotFileLayout) { 
        
               recordInternal(key, snapshot, call, layout) 
        
             } 
        
           } 
        
           class InlineWriteTracker : WriteTracker<CallLocation, LiteralValue<*>>() {

DiskWriteTracker
InlineWriteTracker
ToBeFileWriteTracker (new one)

Help `@ParameterizedTest` users with better error message

At present, a @ParameterizedTest with arguments that in any way affect the appearance of a snapshot fail on the latter iteration(s), due to Selfie wanting the initial snapshot to match all iterations.

It would be neat if this could be detected, and for the runner to then append the additional unique state, instead — either labelled numerically within the snapshot, or maybe numbered + the same argument naming convention as JUnit, i.e. respecting Arguments.of(Named.of(...)), or else falling back to toString().

Disclaimer: I do not know whether JUnit 5 even provides the hooks necessary to see which iteration/argument set is currently executing, but I would hope this is possible.

I'm also totally open to the possibility that you might view this as a test design smell, but I for one get a lot of mileage out of @ParameterizedTests where the snapshot-candidate is not static between iterations.

Ignore comments within String constants

This code is too simple.

selfie/selfie-lib/src/commonMain/kotlin/com/diffplug/selfie/guts/CommentTracker.kt

Lines 51 to 54 in 6c3d0ee

    
           // TODO: there is a bug here due to string constants, and non-C file comments 
        
           val newComment = 
        
               if (str.contains("//selfieonce") || str.contains("// selfieonce")) { 
        
                 WritableComment.ONCE

selfie/selfie-lib/src/commonMain/kotlin/com/diffplug/selfie/guts/SourceFile.kt

Lines 83 to 87 in 6c3d0ee

    
           fun removeSelfieOnceComments() { 
        
             // TODO: there is a bug here due to string constants, and non-C file comments 
        
             contentSlice = 
        
                 Slice(contentSlice.toString().replace("//selfieonce", "").replace("// selfieonce", "")) 
        
           }

For example:

class Test {
  @Test public void example() {
    expectSelfie("underTest").toBe("""
This is a string constant
// selfieonce
The comment above is not really a comment.
We effectively have a "forbidden value" that can't be inside inline snapshots.
     """)
  }
}

String constant too long

We should check the max length of inline strings before writing them.

	fun parseToBe_TODO(lineOneIndexed: Int): ToBeLiteral {
	return parseToBeLike(".toBe_TODO(", lineOneIndexed)
	}
	fun parseToBe(lineOneIndexed: Int): ToBeLiteral {
	return parseToBeLike(".toBe(", lineOneIndexed)
	}
	private fun parseToBeLike(prefix: String, lineOneIndexed: Int): ToBeLiteral {

	fun gradlew(task: String, vararg args: String): AssertionFailedError? {
	return GradleConnector.newConnector()
	.forProjectDirectory(subprojectFolder.parent!!.toFile())
	.connect()
	.use { connection ->

	class DiskWriteTracker : WriteTracker<String, Snapshot>() {
	fun record(key: String, snapshot: Snapshot, call: CallStack, layout: SnapshotFileLayout) {
	recordInternal(key, snapshot, call, layout)
	}
	}

	class InlineWriteTracker : WriteTracker<CallLocation, LiteralValue<*>>() {

	// TODO: there is a bug here due to string constants, and non-C file comments
	val newComment =
	if (str.contains("//selfieonce") \|\| str.contains("// selfieonce")) {
	WritableComment.ONCE

	fun removeSelfieOnceComments() {
	// TODO: there is a bug here due to string constants, and non-C file comments
	contentSlice =
	Slice(contentSlice.toString().replace("//selfieonce", "").replace("// selfieonce", ""))
	}

diffplug / selfie Goto Github PK

selfie's Introduction

Selfie: snapshot testing and memoizing for Java, Kotlin, and the JVM

Key features

Documentation

Contributing

Acknowledgements

selfie's People

Contributors

Stargazers

Watchers

Forkers

selfie's Issues