Need to see actual output of test

[schwa]

When a test fails it would be extremely useful to see what the (unexpected) output actually was.

TAP version 13
1..1
not ok 1 - `radiansToDegrees(degreesToRadians(90.0))` did not produce `Double = 90.0`
  ---
  column: 1
  file: Sources/Everything/Misc/Math.swift
  line: 1
  ...

⏎

[mattt]

Thanks for taking a look at DocTest, @schwa.

The message you're seeing is returned when the REPL produces no output for a statement (if a value is produced, it's included in the message):

DocTest/Sources/DocTest/Statement.swift

Line 42 in 0d68dbe

.failure("- `\(self.code)` did not produce `\(expected)`", directive: nil, metadata: metadata)

That could be made clearer, either in the message itself or by adding "actual" and "expected" keys to the metadata. Any strong feelings one way or another?

Looking at the underlying issue, I'm curious as to why radiansToDegrees didn't produce any value. Could you share that code sample or something else that reproduces this problem?

[schwa]

The function definitely returns a result. But I’ve had general trouble getting doctest working. I’ve not been able to get a test to pass at all. I’ll look into it more and see if I can work out what’s going on.

A -verbose flag emitting way too much information may help people diagnose their own problems.

[mattt]

I have a draft PR with a --verbose flag in #21.

The only blocker is a yak shaving exercise of translating Range<String.Index> values into line / column numbers. Right now, we're printing absolute UTF-16 offsets which aren't particularly helpful. Unless I find existing functionality that does this, I'll probably make something like SourceLocationConverter from SwiftSyntax and release it as a separate package (since this seems generally useful).

[mattt]

@schwa I just tagged a 0.1.0 release with some fixes that I think should help diagnose this issue. When you have a chance, could you please run brew upgrade swift-doctest and re-run swift doctest?

[schwa]

The --verbose flag helped a lot. It's failing to find the tested function even though I'm pointing it at the file containing the function.

I think for the purposes of this bug we can close this issue out - I have the ability to debug test runs now.

I think there may still be issues around how to run doctest on packages etc. I'll poke around more and file anything that comes up.