Feature/merging develop ver.30 by dk1844 · Pull Request #1752 · AbsaOSS/enceladus

dk1844 · 2021-04-15T11:02:00Z

This is essentially merging develop into develop-ver3.0, but since it was non-trivial, this PR was created.

* Create license_check.yml

* Add e2e tests into examples * Fix style * Examples - Fix comments * Apply suggestions from code review Co-authored-by: veprokendlo <68275331+veprokendlo@users.noreply.github.com> * Update vars in test jasons and other small fixes * new test for MappingCR and other small fixes * repair key address * typo remove * add recordId conf into test jasons and update readme Co-authored-by: veprokendlo <68275331+veprokendlo@users.noreply.github.com> Co-authored-by: Veronika Huvarova <vhuvarova@gmail.com> Co-authored-by: Veronika <56251097+HuvarVer@users.noreply.github.com>

* 1648 GroupExplode and Broadcast working with multiple outputs

* #1648 Multiple outputs in UI and additional output encoding

% Conflicts: % data-model/src/main/scala/META-INF/MANIFEST.MF % menas-web/ui/components/dataset/conformanceRule/MappingConformanceRule/targetSchemaFieldSelector.fragment.xml % menas/ui/index.html

benedeki · 2021-04-15T13:46:26Z

Non-trivial 😞
Then I would wait the develop to be updated to the expected (hopefully) version - e.g. merge of #1740

//@dk1844 replying: that's also an option. I felt that there was a lot of added value of creating merge in the first place, hopefully the #1740 addition won't be easier.

…ult values

Zejnilovic · 2021-04-19T10:08:33Z

+ARG DOCKER_BASE_ARTIFACT=nginx
+ARG DOCKER_BASE_TAG=1-alpine
+
+FROM ${DOCKER_BASE_ARTIFACT}:${DOCKER_BASE_TAG}


I know this is a new feature that people are excited about, but I think it was mainly created for multi staged builds.

What does it give us here? What are the pros of this approach? I honestly see only possible problems here + the extra (undeclared) properties in Maven.

if DOCKER_BASE_ARTIFACT and DOCKER_BASE_TAG empty, will be using default values in Dockerfile .
I use these ARG in CodeBuild
DOCKER_BASE_ARTIFACT = ECR/bigdatarnd_tomcat
DOCKER_BASE_TAG = needed version
instead of pulling tomcat from DockerHub , I use an image from ECR. Also it the way to avoid an error about "dockerhub limits" https://docs.docker.com/docker-hub/download-rate-limit/ (In Codebuild I meet the error pretty often )

I would put a comment in there above the ARGVs as to why it is like that.

…d from SparkJobs on execution (#1740) #1702: Different verification of Dataset properties from setup API and from SparkJobs on execution * Essentiality Mandatory received a boolean parameter allowRun * Dataset validation enhanced to support the Mandatory essentiality based on its boolean parameter * Created ValidationKind enumeration with its Spring converter from Spring too * Added query params to dataset validation API to specify how strict the validation should be * Spring RestExceptionHandler changed to return 400 on wrong Query param conversion from string * Essentiality changed to look little more as an enumeration of case objects * String implicits: emptyOrElse, coalesce * UTs * Small code changes * fixed `RunRepositoryIntegrationSuite` not to be local time zone dependent Co-authored-by: Daniel K <dk1844@gmail.com>

…velop-ver.30 # Conflicts: # menas/src/main/scala/za/co/absa/enceladus/menas/MvcConfig.scala

dk1844 · 2021-04-20T06:59:09Z

#1740 merged into develop, updated this PR/branch by merging develop into it.

Zejnilovic

I only went over Docker stuff

…pace trimming (#1761) #1757: Updating Windows Helper scripts for DRA default and CSV whitespace trimming * Windows Helper scripts support CSV whitespace trimming * Windows Helper scripts use DRA as default

sonarqubecloud · 2021-04-20T08:47:52Z

SonarCloud Quality Gate failed.

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
20.6% Duplication

benedeki and others added 7 commits March 16, 2021 01:37

Create license_check.yml (#1724)

d1a860e

* Create license_check.yml

1661 Mapping Multiple Outputs Backend (#1688)

665b34f

* 1648 GroupExplode and Broadcast working with multiple outputs

1662 Mapping Multiple Outputs Frontend (#1721)

d99e57a

* #1648 Multiple outputs in UI and additional output encoding

MANIFEST.MFs LF commit

1c7d8a9

Merge branch 'develop' into develop-ver-3.0

836bbec

% Conflicts: % data-model/src/main/scala/META-INF/MANIFEST.MF % menas-web/ui/components/dataset/conformanceRule/MappingConformanceRule/targetSchemaFieldSelector.fragment.xml % menas/ui/index.html

[develop-ver3.0] mergefix

07d4888

dk1844 requested review from AdrianOlosutean, DzMakatun, HuvarVer, benedeki and lokm01 as code owners April 15, 2021 11:02

dk1844 self-assigned this Apr 15, 2021

dk1844 added the PR:tested Only for PR - PR was tested by a tester (person) label Apr 15, 2021

Zejnilovic and others added 3 commits April 16, 2021 11:17

#1753 Add ignore white spaces option to csv read

2669889

changes in Dockerfile - added new ARGs and in menas new ENV with defa…

8a1c171

…ult values

chnaged docker.properties.template and one more ENV for dockerfile

952e52a

Zejnilovic reviewed Apr 19, 2021

View reviewed changes

AdrianOlosutean and others added 6 commits April 19, 2021 15:42

spark master and password as before

2b0f371

Merge remote-tracking branch 'origin/develop' into feature/merging-de…

43a2740

…velop-ver.30 # Conflicts: # menas/src/main/scala/za/co/absa/enceladus/menas/MvcConfig.scala

integration

90188dc

#1759 Stricter enforcement of DRA in helper scripts

9e2b029

Merge branch 'develop' into feature/merging-develop-ver.30

5d3f4e0

AdrianOlosutean reviewed Apr 20, 2021

View reviewed changes

Comment thread dao/src/main/scala/META-INF/MANIFEST.MF

AdrianOlosutean previously approved these changes Apr 20, 2021

View reviewed changes

SPLINE_URLTEMPLATE moved from docker.prop to Dockerfile

5e90cbd

dvagapov dismissed AdrianOlosutean’s stale review via 5e90cbd April 20, 2021 07:54

dk1844 requested review from AdrianOlosutean and Zejnilovic April 20, 2021 08:02

Zejnilovic previously approved these changes Apr 20, 2021

View reviewed changes

benedeki and others added 2 commits April 20, 2021 10:40

#1757: Updating Windows Helper scripts for DRA default and CSV whites…

715d9ba

…pace trimming (#1761) #1757: Updating Windows Helper scripts for DRA default and CSV whitespace trimming * Windows Helper scripts support CSV whitespace trimming * Windows Helper scripts use DRA as default

Merge branch 'develop' into feature/merging-develop-ver.30

42df938

dk1844 dismissed Zejnilovic’s stale review via 42df938 April 20, 2021 08:46

AdrianOlosutean approved these changes Apr 20, 2021

View reviewed changes

dk1844 merged commit da03de0 into develop-ver-3.0 Apr 20, 2021

dk1844 deleted the feature/merging-develop-ver.30 branch April 20, 2021 09:19

Navigation Menu

Feature/merging develop ver.30 - #1752

Feature/merging develop ver.30#1752
dk1844 merged 19 commits into
develop-ver-3.0from
feature/merging-develop-ver.30

dk1844 commented Apr 15, 2021

Uh oh!

benedeki commented Apr 15, 2021 •

edited by dk1844

Loading

Uh oh!

Uh oh!

Zejnilovic Apr 19, 2021

Uh oh!

dvagapov Apr 19, 2021

Uh oh!

Zejnilovic Apr 20, 2021

Uh oh!

dk1844 commented Apr 20, 2021

Uh oh!

Uh oh!

Zejnilovic left a comment

Uh oh!

sonarqubecloud Bot commented Apr 20, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

dk1844 commented Apr 15, 2021

Uh oh!

benedeki commented Apr 15, 2021 • edited by dk1844 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Zejnilovic Apr 19, 2021

Choose a reason for hiding this comment

Uh oh!

dvagapov Apr 19, 2021

Choose a reason for hiding this comment

Uh oh!

Zejnilovic Apr 20, 2021

Choose a reason for hiding this comment

Uh oh!

dk1844 commented Apr 20, 2021

Uh oh!

Uh oh!

Zejnilovic left a comment

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud Bot commented Apr 20, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

benedeki commented Apr 15, 2021 •

edited by dk1844

Loading