run appium tests in SemaphoreCI (or elsewhere)

[tomholub]

We use SemaphoreCI on all projects, but here for iOS UI tests it was giving us trouble - tests would run successfully on local mac Xcode but not run successfully on CI.

It's hard to say if it's our code or SemaphoreCI at fault without investigating further - so I'll ask Dmitry to look into it.

It may turn out that the issues may be solved by a combination of:

• adjusting the code to run well in CI
• using another CI provider
• using Appium

If it does turn out we should try another CI provider, then here are some options I've seen so far:

• https://www.bitrise.io/why/technologies/ios-continuous-integration (Dmitry mentioned issue with secrets?)
• https://developer.apple.com/documentation/Xcode/About-Continuous-Integration-and-Delivery-with-Xcode-Cloud
• https://circleci.com/docs/2.0/testing-ios/
• https://codemagic.io/ios-continuous-integration/
• GitLab also has CI for iOS but I suppose the GitHub integration may not be very good? https://about.gitlab.com/blog/2016/03/10/setting-up-gitlab-ci-for-ios-projects/

[tomholub]

@fcvakintos seeing your email:

We are looking for another solution regarding UI tests for our iOS application. We made POC for Appium usage so we would like to know if your platform supporting executing Appium tests. If yes could please share documentation of setting up this process. Shortly speaking we are making npm install of Appium library and installing all other node packages which are needed for automation tests. After that all we need is to run the tests which could trigger Appium server process as well.

It's not clear what issue you ran into, which is probably what they'll want to know.

Can you show any error from the SemaphoreCI that you got when using appium? Please make a Draft PR to demostrate the issue you had.

[fcvakintos]

sorry @tomholub I didn't got you correctly before, we didn't run tests on semaphore yet. I will try it

[tomholub]

I thought you had some problem with it. Please try. See if the SemaphoreCI interface allows you to add secret file (env). If not, I'll add it.

[tomholub]

That way we ensure we don't break the tests. For example, #657 is sure to break them.

If they run in CI, the developer making the feature will be responsible for updating them and it will be obvious if they did it successfully or not.

[tomholub]

for semaphoreci can I create new project? cause I feel like I can break existing with my changes
so i can copy ios project and test

There's nothing to break if you work on your own branch then you make a PR and I review it, just like all other changes.

You'll want to add a job here: https://github.com/FlowCrypt/flowcrypt-ios/blob/master/.semaphore/semaphore.yml with the commands to run to install and run appium.

The only change on the actual semaphore project is to add a secrets file. That's not a change that would break anything.

[tomholub]

@fcvakintos what exactly was the problem with secrets on bitrise? They don't have any mechanism for secrets management on the CI?