Add Swift Testing attachments for snapshot failures (Swift 6.2+) #1021
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Create DualAttachment to bridge XCTest and Swift Testing attachments - Store attachment data alongside XCTAttachments for Swift Testing - Record attachments via Swift Testing's Attachment API when available - Add three separate attachments for string diffs (reference, actual, diff) - Optimize large image attachments with automatic compression Addresses pointfreeco#909
- Keep string snapshots with single patch attachment (original behavior) - Only image snapshots create three attachments (reference, failure, difference) - Focus on adding Swift Testing support without changing existing attachment patterns - Add comprehensive tests to verify attachment creation
- Update SwiftTestingAttachmentTests to use withKnownIssue properly for expected failures - Add compiler conditional for inline snapshot test to handle Swift 6.2+ error message format changes - Remove unnecessary @_spi annotations from UIImage snapshots - Update documentation formatting to match existing patterns - Ensure all tests pass with expected known issues
- Match existing codebase pattern using NSLock with @unchecked Sendable - Replace DispatchQueue.sync with lock.lock()/defer { lock.unlock() } - Consistent with Counter class implementation in AssertSnapshot.swift
- Add STAttachments helper to centralize Swift Testing attachment recording - Add proper platform guards matching project conventions for XCTest imports - Simplify nested #if conditions using && operator - Add UUID-based storage for reliable attachment retrieval
Replace brittle AttachmentStorage with direct data embedding in XCTAttachment.userInfo. This eliminates: - Global storage with ObjectIdentifier keys that could collide - Thread synchronization overhead - Complex UUID tracking The new approach embeds image data directly in XCTAttachment's userInfo dictionary, making the data flow direct and collision-free without changing any public APIs.
Author
|
Hey @mbrandonw just noticed you had this branch a while back: Feel free to use this PR as a reference if helpful. 🙂 |
Obbut
reviewed
Sep 24, 2025
Comment on lines
479
to
480
| if let userInfo = attachment.userInfo, | ||
| let imageData = userInfo["imageData"] as? Data |
There was a problem hiding this comment.
@ajkolean I tried your branch but it isn't working for me. I'm snapshotting SwiftUI views as images.
In my case it looks like attachment.userInfo is nil here.
There was a problem hiding this comment.
Same for my case. I'm also not sure if that is the way to access imageData from XCAttachment
Author
There was a problem hiding this comment.
It seems XCTAttachment is deliberately designed as a write-only container. I reconstructed the data rather than trying to extract it from XCTAttachment which should have addressed this.
Author
There was a problem hiding this comment.
Here is a simple Demo that includes that points to this branch, if you clone and run tests couple times, attachments should be visible.
- Make NSImage/UIImage diff() and NSImagePNGRepresentation() internal - Recreate attachment data based on attachment name (reference/failure/difference) - Handle string diffs in default case with difference.patch - Add verification tests that would fail with old userInfo approach Fixes issue where attachment.userInfo was nil, preventing diff attachments from being created for Swift Testing. XCTAttachment doesn't expose its internal data, so we recreate it from the source reference/diffable values. The old code tried to access attachment.userInfo["imageData"] which was always nil because XCTAttachment(image:) doesn't populate userInfo. Our fix uses snapshotting.diffing.toData() to convert reference/diffable values back to Data, and recreates difference images using the now-internal diff() functions. Tests added: - imageDiffCreatesThreeAttachments: Verifies 3 diff attachments are created - stringDiffCreatesOnePatchAttachment: Verifies patch attachment is created - attachmentUserInfoIsNotRequired: Regression test for userInfo approach
- Add file extensions to attachment names (reference.png, failure.png, difference.png) - Use pathExtension from Snapshotting to determine correct extension - Improve performance by reusing already-computed failure message instead of recomputing diff - Update documentation to accurately describe the recreation approach - Add comprehensive documentation to STAttachments helper This fixes the issue where Xcode couldn't display attachment previews because the files had no extension. Now attachments are properly named with extensions based on the snapshot type (png, txt, etc.). Performance improvement: The default case now uses the failure message already computed at line 459 instead of calling snapshotting.diffing.diff() again.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds support for recording snapshot failure artifacts as Swift Testing attachments while maintaining existing XCTest behavior.
Changes
available (canImport(Testing)andcompiler(>=6.2)).UIImage,NSImage) to bundle raw PNG data alongsideXCTAttachmentsso they can be re-recorded under Swift Testing.XCTContextattachments forXCTest.SwiftTestingAttachmentTestssuite to verify attachment behavior for both text and image snapshots.Documentation/SwiftTestingAttachments.mdexplaining feature, usage, and requirements.Why this matters
Adds richer diagnostics for snapshot failures directly in Xcode’s native Swift Testing UI without altering behavior for XCTest users.
Related discussion
See GitHub Discussion #909: “No screenshots attachments in Swift Testing tests” — this PR implements the solution requested there.
Let me know if you want it tweaked further!