Skip to content

add logics to support suborchestration--add one intergration test - #20

Closed
kaibocai (kaibocai) wants to merge 2 commits into
mainfrom
kaibocai/support-suborchestration
Closed

kaibocai (kaibocai) wants to merge 2 commits into
mainfrom
kaibocai/support-suborchestration

Conversation

@kaibocai

@kaibocai kaibocai (kaibocai) commented Mar 30, 2022 •

Copy link
Copy Markdown
Member

This PR contains changes for issue #5 :

  • Add logics to support suborchestration
  • Add one intergration test

this pr based on branch cgillum/error-handling, will wait for pr #19 to merge first, then merge to main branch. Now target to branch cgillum/error-handling for reviewing

Chris Gillum (cgillum) and others added 2 commits March 29, 2022 17:58
- Renamed ErrorDetails to FailureDetails
- Made FailureDetails a property of OrchestrationMetadata
- Removed JSON serialization of errors
- Split error testing into new JUnit class
- Misc. cleanup
return this.callActivity(name, input, Void.class);
}

<V> Task<V> callSubOrchestrator(String name, Object input, String instanceId, Class<V> voidClass);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compared with dotnet, I see there is one more param called Version, Chris Gillum (@cgillum) please let me know if this is necessary. Thank you.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not worry about this for now. I haven't figured out the right way to incorporate version information. Durable Functions for .NET also doesn't support the version parameter.

@kaibocai kaibocai (kaibocai) changed the title add logics to support suborchestration--add one intergration tests add logics to support suborchestration--add one intergration test Mar 30, 2022
Base automatically changed from cgillum/error-handling to main March 31, 2022 01:07

@cgillum Chris Gillum (cgillum) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome progress! A few copy/paste typos that need to be cleaned up. I have a few suggestions as well.

Also, you can go ahead and rebase from main now that my other PR is merged (thanks for reviewing it).

return this.callActivity(name, input, Void.class);
}

<V> Task<V> callSubOrchestrator(String name, Object input, String instanceId, Class<V> voidClass);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not worry about this for now. I haven't figured out the right way to incorporate version information. Durable Functions for .NET also doesn't support the version parameter.

return this.callSubOrchestrator(name, input, null);
}

default <V>Task<V> callSubOrchestrator(String name, Object input, Class<V> voidClass){

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For consistency with activity methods:

Suggested change
default <V>Task<V> callSubOrchestrator(String name, Object input, Class<V> voidClass){
default <V>Task<V> callSubOrchestrator(String name, Object input, Class<V> returnType){

return this.callActivity(name, input, Void.class);
}

<V> Task<V> callSubOrchestrator(String name, Object input, String instanceId, Class<V> voidClass);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Developers can use any class type they want here - not just Void.

Suggested change
<V> Task<V> callSubOrchestrator(String name, Object input, String instanceId, Class<V> voidClass);
<V> Task<V> callSubOrchestrator(String name, Object input, String instanceId, Class<V> returnType);

}

if (instanceId == null) {
instanceId = UUID.randomUUID().toString();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just realized that it can be problematic for us to use random APIs in this code. We should really be using a deterministic UUID generating function. This is another feature that we need to add to the TaskOrchestrationContext class implementation. However, let's not worry about this for now. Can you add a // TODO saying "replace this with a deterministic GUID generation so that it's safe for replay". Maybe we should open an issue on GitHub to track as well so that we don't forget. I think I also need to make this change in .NET.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a bug I created for .NET: microsoft/durabletask-dotnet#9

OrchestratorAction taskAction = this.pendingActions.remove(taskId);
if (taskAction == null) {
String message = String.format(
"Non-deterministic orchestrator detected: a history event scheduling an activity task with sequence ID %d and name '%s' was replayed but the current orchestrator implementation didn't actually schedule this task. Was a change made to the orchestrator code after this instance had already started running?",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"Non-deterministic orchestrator detected: a history event scheduling an activity task with sequence ID %d and name '%s' was replayed but the current orchestrator implementation didn't actually schedule this task. Was a change made to the orchestrator code after this instance had already started running?",
"Non-deterministic orchestrator detected: a history event scheduling a sub-orchestration task with sequence ID %d and name '%s' was replayed but the current orchestrator implementation didn't actually schedule this task. Was a change made to the orchestrator code after this instance had already started running?",

int taskId = subOrchestrationInstanceCompletedEvent.getTaskScheduledId();
TaskRecord<?> record = this.openTasks.remove(taskId);
if (record == null) {
this.logger.warning("Discarding a potentially duplicate TaskCompleted event with ID = " + taskId);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
this.logger.warning("Discarding a potentially duplicate TaskCompleted event with ID = " + taskId);
this.logger.warning("Discarding a potentially duplicate SubOrchestrationInstanceCompleted event with ID = " + taskId);

// TODO: Structured logging
// TODO: Would it make more sense to put this log in the activity executor?
this.logger.fine(() -> String.format(
"%s: Activity '%s' (#%d) completed with serialized output: %s",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"%s: Activity '%s' (#%d) completed with serialized output: %s",
"%s: Sub-orchestrator '%s' (#%d) completed with serialized output: %s",


@ParameterizedTest
@ValueSource(booleans = {true, false})
void activityException(boolean handleException) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest we write an integration test like this one but for sub-orchestrations instead of activities.

@kaibocai

Copy link
Copy Markdown
Member Author

create a new pr here: #21

@kaibocai
kaibocai (kaibocai) deleted the kaibocai/support-suborchestration branch April 2, 2022 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants