[BUG] xcparse doesn't export screenshots for iPhone Xʀ on Xcode 11.1 or below

[vince4]

Describe the bug
xcparse does'nt export screenshots for iPhone Xʀ. It works well for iPhone Xs but when I run xcparse screenshots --model --test-run ./Test-MyProject-2019.11.08_17-11-35-+0100.xcresult/ ./screen/, iPhone Xʀ folder contains empty test configurations folders.

Desktop (please complete the following information):

• OS: Mac OS X 10.15 19A603
• xcparse Version: 1.0.0
• Swift version: 5.1
• XCResult version:

{
  "storage" : {
    "backend" : "fileBacked2",
    "compression" : "standard"
  },
  "dateCreated" : "2019-11-08T17:14:51.000+0100",
  "rootId" : {
    "hash" : "0~HPk8o0sdWVssq8vJoGciopcDFzESyRI2ozvlr7NFNgkOQ9ma1r1HGbYldGtV-QUQLGuYL_oHw6ihqU_O-u4R0A=="
  },
  "externalLocations" : [

  ],
  "version" : {
    "major" : 3,
    "minor" : 21
  }
}

To Reproduce
Steps to reproduce the behavior:

1. Run test with xcodebuild -destination 'platform=iOS Simulator,name=iPhone Xr,OS=13.0' test
2. Export screenshots with xcparse screenshots --model --test-run ./Test-MyProject-2019.11.08_17-11-35-+0100.xcresult/ ./screen/

Expected behavior
Folder iPhone Xʀ containing screenshots

[vince4]

When I use verbose option on xcparse, I see the following command : xcrun xcresulttool export --type file --path /DerivedData/Logs/Test/Test-Transient Testing-2019.11.08_16-52-36-+0100.xcresult' --id 0~n-MS9eN9nOCJMxay0ex48vTab1XSzE5b77M-ZFJuaYpNBx4POYI_zYh0_VXO03gVSn4nFryH4BZbspKbEKXpHQ== --output-path /DerivedData/Logs/Test/screen/iPhone Xʀ/French/Screenshot of main screen (ID 1)_1_622A7AE2-9C08-41D9-BCEC-36183E7FC778.png

So I've tried to run this command by my own and I get Illegal instruction: 4 error.

But If I run the same command putting the destination file in an other folder (
--output-path /DerivedData/Logs/Test/screen/Screenshot of main screen (ID 1)_1_622A7AE2-9C08-41D9-BCEC-36183E7FC778.png)
Then it works !!!

The name iPhone Xʀ seems to be the issue.

[abotkin-cpi]

Thanks for all the details @vince4! I’ll take a look at this today.

[abotkin-cpi]

@vince4 Yep, like you confirmed, this is a crashing issue in xcresulttool in Xcode 11.1 (likely exists in Xcode 11.0 as well but haven't had time to test).

Solution

It appears that this crash in xcresulttool has been resolved by Apple in Xcode 11.2.1 with xcresulttool version 15500. You can check your xcresulttool version with xcrun xcresulttool version. Running Xcode 11.2.1 (11B53) with xcresulttool version 15500, the xcresulttool crash will not occur & the iPhone Xʀ screenshots will come out appropriately.

Folks hitting this should upgrade to Xcode 11.2.1 (opening Xcode and installing the command line tools/required components if prompted) & then re-run xcparse on your xcresult.

How To Diagnose This Affects You

If you're affected by this, you should see a crash log in "~/Library/Logs/DiagnosticReports" on Mojave with a name like "xcresulttool_2019-11-08-131755_abotkin.crash" with a crash signature like this:

Process:               xcresulttool [65386]
Path:                  /Applications/Xcode.app/Contents/Developer/usr/bin/xcresulttool
Identifier:            xcresulttool
Version:               14562
Code Type:             X86-64 (Native)
Parent Process:        ??? [65329]
Responsible:           xcresulttool [65386]
User ID:               502

Date/Time:             2019-11-08 13:17:53.377 -0800
OS Version:            Mac OS X 10.14.6 (18G87)
Report Version:        12
Bridge OS Version:     3.6 (16P6568)
Anonymous UUID:        6FDF86E2-1913-9A67-89F5-56BB3A542ADD

Sleep/Wake UUID:       9E53CE36-6502-4A24-A562-FC490E5D9CA0

Time Awake Since Boot: 340000 seconds
Time Since Wake:       12000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes:       0x0000000000000001, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Illegal instruction: 4
Termination Reason:    Namespace SIGNAL, Code 0x4
Terminating Process:   exc handler [65386]

Application Specific Information:
dyld2 mode

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   xcresulttool                  	0x000000010f6af6da specialized TemporaryFile.init(dir:prefix:suffix:deleteOnClose:) + 1786
1   xcresulttool                  	0x000000010f6842b9 specialized LocalFileSystem.writeFileContents(_:bytes:atomically:) + 137
2   xcresulttool                  	0x000000010f67e8c9 protocol witness for FileSystem.writeFileContents(_:bytes:atomically:) in conformance LocalFileSystem + 9
3   xcresulttool                  	0x000000010f805677 specialized CASResultStore.exportFile(_:to:) + 295
4   xcresulttool                  	0x000000010f80380e protocol witness for ResultReadableStore.exportFile(_:to:) in conformance CASResultStore + 14
5   xcresulttool                  	0x000000010f8037d8 protocol witness for ResultReadableStore.exportFile(_:to:) in conformance CASResultStore + 24
6   xcresulttool                  	0x000000010f7f9040 ResultBundleReader.exportFile(_:to:) + 128
7   xcresulttool                  	0x000000010f7af5c2 Tool._export(stream:) + 1090
8   xcresulttool                  	0x000000010f7bc2c7 Tool.run(outputStream:) + 263
9   xcresulttool                  	0x000000010f8a1262 main + 290
10  libdyld.dylib                 	0x00007fff614a43d5 start + 1

Suggested xcparse Change

The crash report & testing seems to indicate that xcresulttool's with version below 15500 crash if they're given a directory path that contains a non-LATIN1 character such as "iPhone Xʀ". For "iPhone Xʀ", we're filling this in from the model info we see in the xcresult & can workaround this by just modifying the model to "iPhone XR" when running on xcresulttools of that version.

Larger issue is for users who have non-LATIN1 characters in either the file path they want us to export into or in the test run's name (for example, an xcresult with test run configurations named "한국어" & "영어" that the user wants to export to "~/Downloads/아이폰/screenshots". For the test run names, we could replace those with random UUIDs or a lossy ASCII conversion in order to allow the users to get their screenshots & mark that as a warning to the user that we had to do that since they're not on Xcode 11.2.1. For the destination folder having the non-LATIN1 characters, we're just going to need to fail early & tell the user we can't export at all due to the folder name & their Xcode version and to either update their Xcode version or pick a LATIN1 compatible destination.

All these changes should be limited to users who we know have xcresulttool versions below 15500. We should also improve our attachment extraction so that after we see xcresulttool attempt export, we check the file system to see if we see the file. Right now, when xcresulttool crashes, we're not doing any check to see if the file came out and so there's no warning/error thrown to the user.

[abotkin-cpi]

@vince4 A workaround has been implemented in xcparse 1.0.1 to address this issue with Xcode 11.1 and below. Let me know if you see any issues with it. Thanks so much for all the details you put in here; made it a snap!

[vince4]

Thank you @abotkin-cpi for the fix.
I tried with version 1.0.1 and Xcode 11.0 and I get your warning message 👍

Then I installed Xcode 11.2 (without iPhone Xʀ) and it works well.