Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add wait command to CliCore and TemplateProcessor #19

Closed
wants to merge 2 commits into from
Closed

add wait command to CliCore and TemplateProcessor #19

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
589b77f
add wait command to CliCore and TemplateProcessor
Oct 24, 2023
564f117
fix flaky tests
Nov 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 37 additions & 12 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -249,18 +249,19 @@ fact that State-js is written using ES Module syntax.

stated provides a set of REPL commands to interact with the system:

| Command | Description | flags & args | Example |
|----------|----------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------|
| `.init` | Initialize the template from a JSON file. | &bull; `-f <path>` <br> &bull; `--tags=<taglist>`<br>&bull;`--options=<json>` <br> &bull; `--xf=<path>`<br> &bull; `--importPath=<path>` | `.init -f "example/hello.json" --tags=FOO,BAR --xf=~/falken/myEnv.json --options={"strict":{"refs":true}} --importPath=~/falken/mytemplates` |
| `.set` | Set data to a JSON pointer path. | `<path> <data>` | `.set /to "jsonata"` |
| `.from` | Show the dependents of a given JSON pointer. | `<path>` | `.from /a` |
| `.to` | Show the dependencies of a given JSON pointer. | `<path>` | `.to /b` |
| `.in` | Show the input template. | `None` | `.in` |
| `.out` | Show the current state of the template. | `[<jsonPtr>]` | `.out` <br>`.out /data/accounts` |
| `.state` | Show the current state of the template metadata. | `None` | `.state` |
| `.plan` | Show the execution plan for rendering the template. | `None` | `.plan` |
| `.note` | Show a separator line with a comment in the REPL output. | `<comment>` | `.note "Example 8"` |
| `.log` | Set the logging level | `[silent, error, warn, info, verbose, debug]` | `.log silent` |
| Command | Description | flags & args | Example |
|----------|----------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------|
| `.init` | Initialize the template from a JSON file. | &bull; `-f <path>` <br> &bull; `--tags=<taglist>`<br>&bull;`--options=<json>` <br> &bull; `--xf=<path>`<br> &bull; `--importPath=<path>` <br> &bull; `-w 'waitConditionJsonata'` <br> &bull; `-t 'conditionWaitTimeoutMs'` <br> | `.init -f "example/hello.json" --tags=FOO,BAR --xf=~/falken/myEnv.json --options={"strict":{"refs":true}} --importPath=~/falken/mytemplates` |
| `.set` | Set data to a JSON pointer path. | `<path> <data>` | `.set /to "jsonata"` |
| `.from` | Show the dependents of a given JSON pointer. | `<path>` | `.from /a` |
| `.to` | Show the dependencies of a given JSON pointer. | `<path>` | `.to /b` |
| `.in` | Show the input template. | `None` | `.in` |
| `.out` | Show the current state of the template. | `[<jsonPtr>]` | `.out` <br>`.out /data/accounts` |
| `.state` | Show the current state of the template metadata. | `None` | `.state` |
| `.plan` | Show the execution plan for rendering the template. | `None` | `.plan` |
| `.note` | Show a separator line with a comment in the REPL output. | `<comment>` | `.note "Example 8"` |
| `.log` | Set the logging level | `[silent, error, warn, info, verbose, debug]` | `.log silent` |
| `.wait` | Wait for jsonata condition to occur in the template. | &bull; `-w 'waitConditionJsonata'` <br> &bull; `-t 'conditionWaitTimeoutMs'` <br> | `.wait -w foo=bar -t 1500` |


The stated repl lets you experiment with templates. The simplest thing to do in the REPL is load a json file. The REPL
Expand Down Expand Up @@ -1545,6 +1546,30 @@ This can be combined with the `--importPath` option to import files relative to
"res": "bar: foo"
}
```
## Waiting for a Jsonata Condition
To wait for a specific condition to become true, use -w option and an optional timeout (default is 10s).
```json ["error.output.status$=\"counting\"", "status$=\"done\""]
> .init -f example/ex14.yaml -w 'status$="done"' -t 10
zhirafovod marked this conversation as resolved.
Show resolved Hide resolved
{
"error": {
"message": "wait condition status$=\"done\" timed out in 10ms",
"output": {
"incr$": "{function:}",
"counter": 9,
"upCount$": "--interval/timeout--",
"status$": "counting"
}
}
}
> .init -f example/ex14.yaml -w 'status$="done"' -t 150
{
"incr$": "{function:}",
"counter": 11,
"upCount$": "--interval/timeout--",
"status$": "done"
}
```

# Understanding Plans
This information is to explain the planning algorithms to comitters. As a user you do not need to understand how
Stated formulates plans. Before explaining how a plan is made, let's show the end-to-end flow of how a plan is used
Expand Down
6 changes: 3 additions & 3 deletions example/ex14.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"incr": "${function(){$set('/counter',counter+1)}}",
"incr$": "function(){$set('/counter',counter+1)}",
"counter": 0,
"upCount": "${ $setInterval(incr, 1000) }",
"status": "${(counter>10?($clearInterval(upCount);'done'):'counting')}"
"upCount$": "$setInterval(incr$, 10)",
"status$": "counter>10?($clearInterval(upCount$);'done'):'counting'"
}
2 changes: 1 addition & 1 deletion example/ex14.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ incr$: |
$set('/counter',counter+1)
}
counter: 0
upCount$: $setInterval(incr$, 1000)
upCount$: $setInterval(incr$, 10)
status$: |
(
counter>10
Expand Down
31 changes: 16 additions & 15 deletions src/CliCore.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ export default class CliCore {
return path.join(process.cwd(), filepath);
}

//replCmdInoutStr like: -f "example/ex23.json" --tags=["PEACE"] --xf=example/myEnv.json
//replCmdInoutStr like: -f "example/ex23.json" --tags=["PEACE"] --xf=example/myEnv.json -w jsonataExpression -t 1000
async init(replCmdInputStr) {
const parsed = CliCore.parseInitArgs(replCmdInputStr);
const {filepath, tags,oneshot, options, xf:contextFilePath, importPath} = parsed;
Expand All @@ -112,20 +112,8 @@ export default class CliCore {
this.templateProcessor.logger.level = this.logLevel;
this.templateProcessor.logger.debug(`arguments: ${JSON.stringify(parsed)}`);

if (oneshot === true) {
await this.templateProcessor.initialize();
return this.templateProcessor.output;
} else {
try {
await this.templateProcessor.initialize();
return this.templateProcessor.input;
} catch (error) {
return {
name: error.name,
message: error.message
};
}
}
await this.templateProcessor.initialize();
return await this.wait(replCmdInputStr);
}


Expand Down Expand Up @@ -238,5 +226,18 @@ export default class CliCore {
}
return this.templateProcessor.errorReport;
}

// .wait -w jsonataExpression -t 1000
async wait(replCmdInputStr) {
if (!this.templateProcessor) {
throw new Error('Initialize the template first.');
}
const parsed = CliCore.parseInitArgs(replCmdInputStr);
let { w: waitCondition, t: timeout } = parsed;

if (!waitCondition) return this.templateProcessor.input;

return await this.templateProcessor.waitCondition(waitCondition, timeout);
}
}

1 change: 1 addition & 0 deletions src/StatedREPL.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ export default class StatedREPL {
["log", "set the log level [debug, info, warn, error]"],
["debug", "set debug commands (WIP)"],
["errors", "return an error report"],
["wait", '-w jsonataCondition -t 1000 to wait for the condition to return true within 1000ms'],

].map(c=>{
const [cmdName, helpMsg] = c;
Expand Down
85 changes: 84 additions & 1 deletion src/TemplateProcessor.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import ConsoleLogger, {StatedLogger} from "./ConsoleLogger.js";
import FancyLogger from "./FancyLogger.js";
import {LOG_LEVELS} from "./ConsoleLogger.js";
import StatedREPL from "./StatedREPL.js";
import jsonata from "jsonata";



Expand Down Expand Up @@ -1084,9 +1085,26 @@ export default class TemplateProcessor {

setDataChangeCallback(jsonPtr:JsonPointerString, cbFn:(data, ptr:JsonPointerString, removed?:boolean)=>void) {
if(jsonPtr === "/"){
if (this.commonCallback !== undefined) {
return false;
}
this.commonCallback = cbFn;
return true;
}else{
if (this.changeCallbacks.has(jsonPtr)) {
return false
}
this.changeCallbacks.set(jsonPtr, cbFn);
return true;
}
return false;
}

removeDataChangeCallback(jsonPtr) {
if(jsonPtr === "/"){
this.commonCallback = undefined;
}else if (this.changeCallbacks.has(jsonPtr)) {
this.changeCallbacks.delete(jsonPtr);
}
}

Expand All @@ -1111,6 +1129,72 @@ export default class TemplateProcessor {
return null;
}


async waitCondition(waitCondition, timeout = 10000) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

waitCondition and timeout arguments should have types. Return type should be explicit. Whenever we add a new method we should try to use TS best practices.

return new Promise((resolve, reject) => {
const startTime = Date.now();
let conditionExpression;
let timeoutId; // Declare timeoutId here

try {
conditionExpression = jsonata(waitCondition);
} catch (e) {
this.logger.error(`invalid wait condition expression: ${waitCondition}, ${e}`);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure this is the right approach. I try to wrap functions in safe() so that I can simply throw error and rely on the framework to catch it and pack it into an {error:{...}} object

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried a few different approaches. I like with withErrorHandling to handling calling to an external function, where we don't know the context. In this case, I am handling errors from my own code execution, and I think we can do better with a custom error messages.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but your custom error message will be used by withErrorHandling. All you need to do is throw an error:

private withErrorHandling(fn) {
        return (...args) => {
            try {
                const result = fn(...args);
                if (result instanceof Promise) {
                    return result.catch(error => {
                        this.logger.error(error.toString());
                        return {
                            "error": {
                                message: error.message,
                                name: error.name,
                                stack: error.stack,
                            }
                        };
                    });
                }
                return result;
            } catch (error) {
                this.logger.error(error.toString());
                return {
                    "error": {
                        message: error.message,
                        name: error.name,
                        stack: error.stack,
                    }
                };
            }
        };
    };

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^^^ see how withErrorHandling preserves these properties. So why don't you just catch(e) and set it's message, and rethrow it?

resolve({
"error": {
message: e.message,
name: e.name,
stack: e.stack,
input: this.input
}
});
}

const checkTimeout = () => {
if (Date.now() - startTime >= timeout) {
this.removeDataChangeCallback('/');
this.logger.debug(`wait condition ${waitCondition} timed out in ${timeout}ms`);
resolve({
"error": {
message: `wait condition ${waitCondition} timed out in ${timeout}ms`,
output: JSON.parse(StatedREPL.stringify(this.output)) // deep copy
}
});
} else {
timeoutId = setTimeout(checkTimeout, 100); // save timer id so we can cancel it later
}
};

let checkConditionCallback = async (data, jsonPtr) => {
let matchedCondition = await conditionExpression.evaluate(this.output);
if (matchedCondition===true) {
zhirafovod marked this conversation as resolved.
Show resolved Hide resolved
this.removeDataChangeCallback('/');
clearTimeout(timeoutId); // Clear the timeout when condition is met
this.logger.debug(`received data change matching ${waitCondition} for ${jsonPtr} with data ${data}`);
resolve(JSON.parse(StatedREPL.stringify(this.output))); // deep copy
} else {
this.logger.debug(`received data change not matching ${waitCondition} for ${jsonPtr} with data ${data}`);
}
};

checkConditionCallback = checkConditionCallback.bind(this);
const callbackIsSet = this.setDataChangeCallback("/", checkConditionCallback);
if(callbackIsSet === false){
clearTimeout(timeoutId);
resolve({"error": {
message: "can't use wait condition because a callback is already set"
}});
}

//execute the callback once to evaluate if the condition is already met
if (this.isInitializing === false) {
checkConditionCallback({}, "/");
}

timeoutId = setTimeout(checkTimeout, 100); // Assign the timeout ID here
});
}

private async localImport(filePathInPackage) {
// Resolve the package path
const {importPath} = this.options;
Expand Down Expand Up @@ -1138,6 +1222,5 @@ export default class TemplateProcessor {
}
return content;
}

}

6 changes: 6 additions & 0 deletions src/TestUtils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,12 @@ export function runMarkdownTests(testData, cliInstance, printFunction = StatedRE

if (jsonataExpr) {
const compiledExpr = jsonata(jsonataExpr);
const result = await compiledExpr.evaluate(responseNormalized);
if (result !== true) {
// If the result is not true, throw an error with details
throw new Error(`JSONata Test Failed: Expected JSONata expression to evaluate to true.\nExpression: ${jsonataExpr}\nResponse: ${JSON.stringify(responseNormalized, null, 2)}\nEvaluation result: ${result}`);
}
expect(result).toBe(true);
expect(await compiledExpr.evaluate(responseNormalized)).toBe(true);
} else {
const expected = expectedResponse ? JSON.parse(expectedResponse) : undefined;
Expand Down
Loading
Loading