pointfreeco / swift-macro-testing

Magical testing tools for Swift macros.

🎬 www.pointfree.co

This repo contains the full source code for the Point-Free website, a video series exploring advanced programming topics in Swift. The codebase is split into 3 pieces:

• PointFree: This is the core application, and is responsible for routing requests, loading data and rendering HTML and CSS.
• Styleguide: This library contains functions and data types for creating a consistent style across the entire website.
• Server: This is the actual executable server. It uses NIO to handle the low-level server responsibilities, and hands everything else over to the PointFree package.

Getting Started

Interested in a video tour of the code base?

The repo contains an extensive test suite and some playgrounds to explore. To get things running:

• Open up a terminal window and grab the code:

  git clone https://github.com/pointfreeco/pointfreeco.git
  cd pointfreeco

• Make sure cmark is installed. You can install it with Homebrew:

  brew install cmark # or your preferred installation method

• Make sure Postgres is installed and running. It's our database of choice. You can install it with Homebrew:

  brew install postgres # or your preferred installation method
  brew services start postgresql # or your preferred launch method
  make db

  (If you use Postgres.app, EnterpriseDB, or another installation method, please follow some additional instructions in the CPostgreSQL README.)

With the project open in Xcode, you can:

• Run the server locally
  • Select the Server target
  • Run: Command+R
  • Visit http://localhost:8080
• Explore our playgrounds
  • Select the PointFree target
  • Build: Command+B
  • Open a playground!

Some fun things to explore

There're a lot of fun things to explore in this repo. For example:

• We develop web pages in playgrounds for a continuous feedback loop. This is made possible by the fact that the entire server stack is composed of pure functions with side-effects pushed to the boundaries of the application. It allows us to load up any request in isolation, including POST requests, all without ever worrying about doing a side-effect.

• We use snapshot testing to capture full data structures in order to verify their correctness. Not only do we do this in the traditional way of taking screenshots of web pages at various break points (e.g. on iPhone and desktop), but we can also snapshot any entire request-to-response lifecycle (e.g. the POST to a signup page does the correct redirect).

▿ Step
  ResponseEnded

▿ Request
  POST http://localhost:8080/launch-signup

  [email protected]

▿ Response
  Status 302 FOUND
  Location: /?success=true

Xcode Color Theme

Like the color theme we use in our episodes? Run make colortheme to install locally!

Related projects

Point-Free uses a bunch of interesting open-source software:

• 🗺 swift-html: A Swift DSL for type-safe, extensible, and transformable HTML documents.
• 🕸 swift-web: A collection of types and functions for dealing with common web server concerns, such as HTML render, CSS preprocessing, middleware and more.
• 🎶 swift-prelude: Offers a standard library for experimental functional programming in Swift.
• 🏷 swift-tagged: Helps us create strong contracts with our data boundaries, like JSON from GitHub and Stripe, and our PostgreSQL data.
• 📸 swift-snapshot-testing: Powers our testing infrastructure by taking snapshots of various data structures to guarantee the correctness of their output. We use this on everything from middleware to ensure requests are correctly transformed into responses, and even entire web pages to make sure the site looks correct at a variety of sizes (e.g. on iPhone and desktop).

Explore more of our open-source on the Point-Free organization.

Learn More

Brandon gave a talk about most of the core ideas that went into this project at Swift Summit 2017.

Find this interesting?

Then check out Point-Free!

License

The content of this project itself is licensed under the CC BY-NC-SA 4.0 license, and the underlying source code used to format and display that content is licensed under the MIT license.

swift-macro-testing's Issues

Fatal error on incorrect usage

Description

On swift-macro-testing 0.2.0 and main, the following error in assertMacro usage results in a crash:

    assertMacro {
      """
      @MemberwiseInit
      public struct Pedometer {
        var stepsToday = 0
      }
      """
    } expansion: {
      """
      @MemberwiseInit
      public struct Pedometer {
        var stepsToday = 0
            ┬─────────────
            ╰─ 🛑 @MemberwiseInit requires explicit type declarations for `var` stored properties.
      }
      """
    }

My user error was to use the label expansion: instead of diagnostics:. The crash happens deep into the SyntaxVisitor.walk due to an unsafe unwrap. Every one of my tests exercising a diagnostic had this user error and resulted in the same crash.

I discovered this while updating my tests to 0.2.0, using an overly naive find/replace from "matches" to "expansion", having eschewed the given "re-record" advice.

This strikes me as a Swift Syntax issue: I don't think Swift Syntax's RawSyntaxChildren is ever intended to ever fatal error, even on bad input. Thoughts?

Checklist

• If possible, I've reproduced the issue using the main branch of this package.
• This issue hasn't been addressed in an existing GitHub issue or discussion.

Expected behavior

No response

Actual behavior

No response

Steps to reproduce

No response

swift-macro-testing version information

0.2.0, main (3c9424a)

Destination operating system

No response

Xcode version information

Xcode Version 15.0 (15A240d)

Swift Compiler version information

No response

The documentation Link on README does not work.

Description

When clicking the Documentation link on README, we've got 404 error.
The link URL is https://pointfreeco.github.io/swift-macro-testing/main/documentation/macrotesting.

I noticed it since I found this page from the comment. If the reason of the error is simply because the link has not been ready yet, please close this issue.

Checklist

• If possible, I've reproduced the issue using the main branch of this package.
• This issue hasn't been addressed in an existing GitHub issue or discussion.

Expected behavior

We can move to the proper documentation page.

Actual behavior

We've got 404 error.

Steps to reproduce

No response

swift-macro-testing version information

No response

Destination operating system

No response

Xcode version information

No response

Swift Compiler version information

No response

Exclusively applicable fix-its aren't testable

Description

I have a diagnostic with several fix-its, where only one can be applied. Here's the usage, and note that, while diagnostics is correct and (very!) useful, both fixes and expansion are nonsensical:

assertMacro {
  """
  @MemberwiseInit
  struct S {
    @Init(default: "Blob") var name = "Foo"
  }
  """
} diagnostics: {
  """
  @MemberwiseInit
  struct S {
    @Init(default: "Blob") var name = "Foo"
          ┬──────────────
          ╰─ 🛑 Custom 'default' can't be applied to already initialized variable
             ✏️ Remove 'default: "Blob"'
             ✏️ Remove '@Init(default: "Blob")'
             ✏️ Remove '= "Foo"'
  }
  """
} fixes: {
  """
  @MemberwiseInit
  struct S {
    @Init
  }
  """
} expansion: {
  """
  struct S {
    @Init

    internal init() {
    }
  }
  """
}

To show the actual fix-it behavior applied in Xcode, here's a theoretical array syntax format:

…
} fixes: {
  [
    """
    @MemberwiseInit
    struct S {
      @Init var name = "Foo"
    }
    """,
    """
    @MemberwiseInit
    struct S {
      var name = "Foo"
    }
    """,
    """
    @MemberwiseInit
    struct S {
      @Init(default: "Blob") var name: String
    }
    """,
  ]
} expansion: {
…

And, a theoretical dictionary syntax format:

} fixes: {
  [
    #"✏️ Remove 'default: "Blob"'"#:
    """
    @MemberwiseInit
    struct S {
      @Init var name = "Foo"
    }
    """,
    #"✏️ Remove '@Init(default: "Blob")'"#:
    """
    @MemberwiseInit
    struct S {
      var name = "Foo"
    }
    """,
    #"✏️ Remove '= "Foo"'"#:
    """
    @MemberwiseInit
    struct S {
      @Init(default: "Blob") var name: String
    }
    """,
  ]
} expansion: {

What do you think?

P.S.: swift-macro-testing is great—I'm heavily using/abusing it. Thank you!

Checklist

• If possible, I've reproduced the issue using the main branch of this package.
• This issue hasn't been addressed in an existing GitHub issue or discussion.

Expected behavior

Each fix-it should be applied to the original source independently, each producing a corresponding fixed-source string.

Actual behavior

It's assumed that all fix-its can always be applied on top of each other, accumulating into a single string.

Steps to reproduce

No response

swift-macro-testing version information

0.2.2

Destination operating system

macOS 14.1.2

Xcode version information

Xcode Version 15.0.1 (15A507)

Swift Compiler version information

swift-driver version: 1.87.1 Apple Swift version 5.9 (swiftlang-5.9.0.128.108 clang-1500.0.40.1)
Target: arm64-apple-macosx14.0

Fix-it coverage lost when macros include diagnostics with and without fix-its

Description

assertMacro only applies fix-its when all emitted diagnostics have one or more fix-its, so fix-it coverage is lost when macros include diagnostics with and without fix-its. Instead, fixes should be included if any diagnostic has fix-its, even though it’s probable that the fixed source won’t resolve all the diagnostics. (I say “probable” because it is possible, yet unusual, for one fix-it to resolve multiple diagnostics.)

Currently, the following test:

func testInvalidLabelAndDefaultOnMultipleBindings() {
  assertMacro {
    """
    @MemberwiseInit
    struct S {
      @Init(default: 0, label: "$foo") let x, y: T
    }
    """
  }
}

records with just diagnostics:

func testInvalidLabelAndDefaultOnMultipleBindings() {
  assertMacro {
    """
    @MemberwiseInit
    struct S {
      @Init(default: 0, label: "$foo") let x, y: T
    }
    """
  } diagnostics: {
    """
    @MemberwiseInit
    struct S {
      @Init(default: 0, label: "$foo") let x, y: T
                        ┬────────────
            │           ╰─ 🛑 Custom 'label' can't be applied to multiple bindings
            ┬──────────
            ╰─ 🛑 Custom 'default' can't be applied to multiple bindings
               ✏️ Remove 'default: 0'
    }
    """
  }
}

I'd expect that the following fixes closure would be included after diagnostics:

[…]
  } fixes: {
    """
    @MemberwiseInit
    struct S {
      @Init(label: "$foo") let x, y: T
    }
    """
  }

Checklist

• If possible, I've reproduced the issue using the main branch of this package.
• This issue hasn't been addressed in an existing GitHub issue or discussion.

Expected behavior

No response

Actual behavior

No response

Steps to reproduce

No response

swift-macro-testing version information

0.2.0, main (15916c0)

Destination operating system

macOS 14.2

Xcode version information

Xcode Version 15.1 (15C65)

Swift Compiler version information

swift-driver version: 1.87.3 Apple Swift version 5.9.2 (swiftlang-5.9.2.2.56 clang-1500.1.0.2.5)
Target: arm64-apple-macosx14.0

Fix-it targeting first child node instead replaces parent syntax collection

Description

When assertMacro applies a fix-it intended to replace the first node within a collection syntax type, the collection node is replaced instead. This results in the loss of nodes that were not targeted by the fix-it.

Checklist

• If possible, I've reproduced the issue using the main branch of this package.
• This issue hasn't been addressed in an existing GitHub issue or discussion.

Expected behavior

This original source:

@ReplaceFirstMember
struct FooBar {
  let foo: Foo
  let bar: Bar
  let baz: Baz
}

should have the following snapshotted fixes content:

@ReplaceFirstMember
struct FooBar {
  let oye: Oye
  let bar: Bar
  let baz: Baz
}

Actual behavior

Incorrect snapshotted fixes content:

@ReplaceFirstMember
struct FooBar {
  let oye: Oye
}

Steps to reproduce

1. Create a struct with multiple members.
2. Apply a fix-it to replace the first member of the struct.
3. Observe that all members are replaced, not just the targeted member.

Here's a test that reproduces the problem. Note the content of fixes:

import MacroTesting
import SwiftDiagnostics
import SwiftSyntax
import SwiftSyntaxBuilder
import SwiftSyntaxMacros
import XCTest

fileprivate enum ReplaceFirstMemberMacro: MemberMacro {
  public static func expansion(
    of node: AttributeSyntax,
    providingMembersOf declaration: some DeclGroupSyntax,
    in context: some MacroExpansionContext
  ) throws -> [DeclSyntax] {
    guard
      let nodeToReplace = declaration.memberBlock.members.first,
      let newNode = try? MemberBlockItemSyntax(
        decl: VariableDeclSyntax(SyntaxNodeString(stringLiteral: "\n  let oye: Oye"))
      )
    else { return [] }

    context.diagnose(
      Diagnostic(
        node: node.attributeName,
        message: SimpleDiagnosticMessage(
          message: "First member needs to be replaced",
          diagnosticID: MessageID(domain: "domain", id: "diagnostic2"),
          severity: .warning
        ),
        fixIts: [
          FixIt(
            message: SimpleDiagnosticMessage(
              message: "Replace the first member",
              diagnosticID: MessageID(domain: "domain", id: "fixit1"),
              severity: .error
            ),
            changes: [
              .replace(oldNode: Syntax(nodeToReplace), newNode: Syntax(newNode))
            ]
          )
        ]
      )
    )

    return []
  }
}

final class ReplaceMemberMacroTests: BaseTestCase {
  override func invokeTest() {
    withMacroTesting(macros: [ReplaceFirstMemberMacro.self]) {
      super.invokeTest()
    }
  }

  func testReplaceFirstMember() {
    assertMacro {
      """
      @ReplaceFirstMember
      struct FooBar {
        let foo: Foo
        let bar: Bar
        let baz: Baz
      }
      """
    } diagnostics: {
      """
      @ReplaceFirstMember
       ┬─────────────────
       ╰─ ⚠️ First member needs to be replaced
          ✏️ Replace the first member
      struct FooBar {
        let foo: Foo
        let bar: Bar
        let baz: Baz
      }
      """
    } fixes: {
      """
      @ReplaceFirstMember
      struct FooBar {
        let oye: Oye
      }
      """
    } expansion: {
      """
      struct FooBar {
        let oye: Oye
      }
      """
    }
  }
}

swift-macro-testing version information

0.2.0, main (6e93798)

Destination operating system

macOS 14.2

Xcode version information

Xcode Version 15.1 (15C65)

Swift Compiler version information

Apple Swift version 5.9.2 (swiftlang-5.9.2.2.56 clang-1500.1.0.2.5)
Target: arm64-apple-macosx14.0

Generated expansion includes unexpected newlines when using Quick

Description

Thank you for creating this library! It fills a sorely-needed gap in the new compiler macro feature.

I'm writing BDD tests using the Quick framework. When recording a new assertion, I see the following result:

import MacroTesting
import Quick

@testable import MyMacroImplementation

class MacroSpec: QuickSpec {
    
    override func invokeTest() {
        withMacroTesting(macros: ["MyMacro": MyMacro.self]) {
            super.invokeTest()
        }
    }
    
    override class func spec() {
        describe("the @MyMacro macro") {
            it("should fail when annotating a non-class type") {
                assertMacro {
                    """
                    @MyMacro()
                    struct S {}
                    """
                } diagnostics: {
                    
"""
                    
@MyMacro()
                    
╰─ 🛑 'MyMacro' macro can only be applied to a class
                    
struct S {}
                    
"""
                }
            }
            
        
        }
    }
    
}

Beyond just the initial formatting (non-indented), this generated expansion will fail because of the newlines between each generated statement. This example demonstrates a generated diagnostic message, but I've also observed this for the "expansion" argument as well.

Checklist

• If possible, I've reproduced the issue using the main branch of this package.
• This issue hasn't been addressed in an existing GitHub issue or discussion.

Expected behavior

Given the above example, the generated diagnostic string should look like the following (indentation omitted):

"""
@MyMacro()
╰─ 🛑 'MyMacro' macro can only be applied to a class
struct S {}
"""

Actual behavior

The generated string contains superfluous newlines:

"""
                    
@MyMacro()
                    
╰─ 🛑 'MyMacro' macro can only be applied to a class
                    
struct S {}
                    
"""

Steps to reproduce

1. Create a new Swift package with test dependencies on Quick and swift-macro-testing
2. Create any valid macro and associated test class
3. Create a test method using assertMacro to test the given macro, only populate the first closure
4. Run the test to generate the "recording"

swift-macro-testing version information

0.2.1

Destination operating system

macOS 13

Xcode version information

15.0 (15A240d)

Swift Compiler version information

swift-driver version: 1.87.1 Apple Swift version 5.9 (swiftlang-5.9.0.128.108 clang-1500.0.40.1)
Target: arm64-apple-macosx13.0