Add lyng.io.process module for process execution utilities and shell scripting sugar (e.g., sh, exec, CommandRun).
This commit is contained in:
parent
3e74019d9d
commit
234d1ef02b
@ -88,6 +88,10 @@ Sources: `lynglib/src/commonMain/kotlin/net/sergeych/lyng/Script.kt`, `lynglib/s
|
|||||||
Requires installing `lyngio` into the import manager from host code.
|
Requires installing `lyngio` into the import manager from host code.
|
||||||
- `import lyng.io.fs` (filesystem `Path` API)
|
- `import lyng.io.fs` (filesystem `Path` API)
|
||||||
- `import lyng.io.process` (process execution API)
|
- `import lyng.io.process` (process execution API)
|
||||||
|
- Shell-script sugar: `sh(command): CommandRun` and `exec(executable, args=[]): CommandRun`.
|
||||||
|
- Prefer `sh("git status --short").out` for small shell output, `sh("...").lines` for large stdout streams, and `.check()` for commands that must exit with code 0.
|
||||||
|
- Prefer `exec("git", ["add", file])` when arguments come from data, filenames, or user input; it bypasses shell parsing.
|
||||||
|
- `CommandRun` is active and owns process pipes; choose one consumption path per stream (`out` or `lines`, `err` or `errorLines`).
|
||||||
- `import lyng.io.console` (console capabilities, geometry, ANSI/output, events)
|
- `import lyng.io.console` (console capabilities, geometry, ANSI/output, events)
|
||||||
- `import lyng.io.http` (HTTP/HTTPS client API)
|
- `import lyng.io.http` (HTTP/HTTPS client API)
|
||||||
- `import lyng.io.http.server` (minimal HTTP/1.1 and WebSocket server API)
|
- `import lyng.io.http.server` (minimal HTTP/1.1 and WebSocket server API)
|
||||||
|
|||||||
@ -3,6 +3,7 @@
|
|||||||
This module provides a way to run external processes and shell commands from Lyng scripts. It is designed to be multiplatform and uses coroutines for non-blocking execution.
|
This module provides a way to run external processes and shell commands from Lyng scripts. It is designed to be multiplatform and uses coroutines for non-blocking execution.
|
||||||
|
|
||||||
> **Note:** `lyngio` is a separate library module. It must be explicitly added as a dependency to your host application and initialized in your Lyng scopes.
|
> **Note:** `lyngio` is a separate library module. It must be explicitly added as a dependency to your host application and initialized in your Lyng scopes.
|
||||||
|
> The reference `lyng`/`jlyng` CLI installs this module in its import manager, so CLI scripts can use `import lyng.io.process` directly.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -44,24 +45,241 @@ suspend fun bootstrapProcess() {
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
#### Using from Lyng scripts
|
#### Shell-script shorthand
|
||||||
|
|
||||||
|
Most CLI scripts should start with the shorthand API:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
import lyng.io.process
|
||||||
|
|
||||||
|
val branch = sh("git branch --show-current").out.trim()
|
||||||
|
println("Branch: " + branch)
|
||||||
|
```
|
||||||
|
|
||||||
|
`sh(command)` starts `command` through the platform shell and returns an active `CommandRun`.
|
||||||
|
The process is already running when the handle is returned. The handle gives you both convenient capture properties and streaming line flows:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
val run = sh("git status --short")
|
||||||
|
|
||||||
|
println(run.out) // capture all stdout as String
|
||||||
|
println(run.err) // capture all stderr as String
|
||||||
|
println(run.code) // known after out/err/wait/check, otherwise null
|
||||||
|
println(run.ok) // true, false, or null
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `.check()` when a non-zero exit code should fail the script:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
sh("lyng fmt --check examples/*.lyng").check()
|
||||||
|
sh("git diff --check").check()
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `.wait()` when the exit code is data and should not raise:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
val code = sh("git diff --quiet").wait()
|
||||||
|
if (code == 0) {
|
||||||
|
println("clean")
|
||||||
|
} else {
|
||||||
|
println("changed")
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `.lines` for output that may be large. It streams stdout instead of buffering the entire output:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
for (file in sh("git ls-files").lines) {
|
||||||
|
if (file.endsWith(".lyng")) {
|
||||||
|
println(file)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
You can combine it with normal iterable helpers:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
val count = sh("git ls-files").lines.count {
|
||||||
|
it.endsWith(".lyng")
|
||||||
|
}
|
||||||
|
|
||||||
|
println("Lyng files: " + count)
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `exec(executable, args)` when arguments come from data, filenames, or user input:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
val file = ARGV[0] as String
|
||||||
|
exec("git", ["add", file]).check()
|
||||||
|
```
|
||||||
|
|
||||||
|
`exec(...)` bypasses the shell, so arguments are passed as argv entries rather than parsed by shell quoting rules.
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
val status = exec("git", ["status", "--short"]).out
|
||||||
|
println(status)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### Capture vs streaming
|
||||||
|
|
||||||
|
`CommandRun` is active: it owns a running process. Choose the consumption style intentionally.
|
||||||
|
|
||||||
|
- `.out` captures stdout into memory, waits for the process to finish, and also drains stderr concurrently.
|
||||||
|
- `.err` captures stderr into memory, waits for the process to finish, and also drains stdout concurrently.
|
||||||
|
- `.check()` captures both streams, waits, and fails if the exit code is non-zero.
|
||||||
|
- `.lines` streams stdout line by line and does not buffer the whole output.
|
||||||
|
- `.errorLines` streams stderr line by line and does not buffer the whole error output.
|
||||||
|
- `.wait()` only waits and returns the exit code.
|
||||||
|
|
||||||
|
Treat a `CommandRun` as a one-shot stream owner. Pick one stdout consumption path (`out` or `lines`) and one stderr consumption path (`err` or `errorLines`). Do not collect the same stream twice; it is backed by the process pipe.
|
||||||
|
|
||||||
|
For small output, capture is simplest:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
val version = sh("git describe --tags --always").out.trim()
|
||||||
|
```
|
||||||
|
|
||||||
|
For large output, stream:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
for (line in sh("find . -type f").lines) {
|
||||||
|
if (line.endsWith(".lyng")) {
|
||||||
|
println(line)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
For commands that may write a lot to both stdout and stderr, consume both streams. One practical pattern is to collect stderr in a background task while processing stdout:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
val r = sh("some-command-that-writes-a-lot")
|
||||||
|
|
||||||
|
val errors: List<String> = []
|
||||||
|
val stderrTask = launch {
|
||||||
|
for (line in r.errorLines) {
|
||||||
|
errors.add(line)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for (line in r.lines) {
|
||||||
|
println("OUT: " + line)
|
||||||
|
}
|
||||||
|
|
||||||
|
val code = r.wait()
|
||||||
|
stderrTask.await()
|
||||||
|
|
||||||
|
if (code != 0) {
|
||||||
|
println(errors.joinToString("\n"))
|
||||||
|
exit(code)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
As with ordinary shell programming, avoid using `.out` for unbounded output. It is meant for command results that comfortably fit in memory.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### Common script patterns
|
||||||
|
|
||||||
|
Capture one small value:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
val currentCommit = exec("git", ["rev-parse", "--short", "HEAD"]).out.trim()
|
||||||
|
```
|
||||||
|
|
||||||
|
Run a command for its side effects and fail on error:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
exec("mkdir", ["-p", "build/reports"]).check()
|
||||||
|
sh("lyng fmt --check src/**/*.lyng").check()
|
||||||
|
```
|
||||||
|
|
||||||
|
Filter a long output stream:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
for (path in sh("git ls-files").lines) {
|
||||||
|
if (path.endsWith(".lyng")) {
|
||||||
|
println(path)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Branch on a command status:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
if (exec("git", ["diff", "--quiet"]).wait() != 0) {
|
||||||
|
println("working tree has changes")
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Keep user-controlled text out of the shell:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
val file = ARGV[0] as String
|
||||||
|
exec("git", ["log", "--oneline", "--", file]).check()
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### `sh` vs `exec`
|
||||||
|
|
||||||
|
Use `sh(...)` when you intentionally want shell syntax:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
sh("git status --short | wc -l").out.trim()
|
||||||
|
sh("find . -name '*.lyng' | sort").check()
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `exec(...)` when you already have structured arguments:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
exec("git", ["commit", "-m", message]).check()
|
||||||
|
exec("cp", [sourcePath, targetPath]).check()
|
||||||
|
```
|
||||||
|
|
||||||
|
This distinction matters for safety. Shell commands are parsed by `/bin/sh -c` on Unix-like systems or the platform shell on supported platforms. If you interpolate untrusted text into a shell string, it can change the command being run. Prefer `exec(...)` for user input and filenames.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### Low-level process API
|
||||||
|
|
||||||
|
The shorthand API is built on top of `Process.execute(...)` and `Process.shell(...)`. Use the low-level API when you need direct process control, including explicit signals or forceful termination:
|
||||||
|
|
||||||
```lyng
|
```lyng
|
||||||
import lyng.io.process
|
import lyng.io.process
|
||||||
|
|
||||||
// Execute a process with arguments
|
// Execute a process with arguments
|
||||||
val p = Process.execute("ls", ["-l", "/tmp"])
|
val p = Process.execute("ls", ["-l", "/tmp"])
|
||||||
for (line in p.stdout) {
|
for (line in p.stdout()) {
|
||||||
println("OUT: " + line)
|
println("OUT: " + line)
|
||||||
}
|
}
|
||||||
val exitCode = p.waitFor()
|
val exitCode = p.waitFor()
|
||||||
println("Process exited with: " + exitCode)
|
println("Process exited with: " + exitCode)
|
||||||
|
|
||||||
// Run a shell command
|
// Run a shell command
|
||||||
val sh = Process.shell("echo 'Hello from shell' | wc -w")
|
val shellProcess = Process.shell("echo 'Hello from shell' | wc -w")
|
||||||
for (line in sh.stdout) {
|
for (line in shellProcess.stdout()) {
|
||||||
println("Word count: " + line.trim())
|
println("Word count: " + line.trim())
|
||||||
}
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Signals and termination:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
val server = Process.shell("python3 -m http.server 8080")
|
||||||
|
delay(1000)
|
||||||
|
server.signal("SIGTERM")
|
||||||
|
val code = server.waitFor()
|
||||||
|
println("server exited: " + code)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#### Platform information
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
import lyng.io.process
|
||||||
|
|
||||||
// Platform information
|
// Platform information
|
||||||
val details = Platform.details()
|
val details = Platform.details()
|
||||||
@ -77,15 +295,57 @@ if (Platform.isSupported()) {
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
#### Executable script example
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
#!/usr/bin/env lyng
|
||||||
|
|
||||||
|
import lyng.io.process
|
||||||
|
|
||||||
|
if (ARGV.size == 0) {
|
||||||
|
println("usage: changed-files.lyng <extension>")
|
||||||
|
exit(2)
|
||||||
|
}
|
||||||
|
|
||||||
|
val ext = ARGV[0] as String
|
||||||
|
var matches = 0
|
||||||
|
|
||||||
|
for (file in sh("git status --short").lines) {
|
||||||
|
if (file.endsWith(ext)) {
|
||||||
|
println(file)
|
||||||
|
matches++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
println("matched: " + matches)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
#### API Reference
|
#### API Reference
|
||||||
|
|
||||||
##### `Process` (static methods)
|
##### `Process` (static methods)
|
||||||
- `execute(executable: String, args: List<String>): RunningProcess` — Start an external process.
|
- `execute(executable: String, args: List<String>): RunningProcess` — Start an external process.
|
||||||
- `shell(command: String): RunningProcess` — Run a command through the system shell (e.g., `/bin/sh` or `cmd.exe`).
|
- `shell(command: String): RunningProcess` — Run a command through the system shell (e.g., `/bin/sh` or `cmd.exe`).
|
||||||
|
|
||||||
|
##### Shorthand functions
|
||||||
|
- `sh(command: String): CommandRun` — Run a command through the system shell and return an active handle.
|
||||||
|
- `exec(executable: String, args: List<String> = []): CommandRun` — Run an executable with argv-style arguments and return an active handle.
|
||||||
|
|
||||||
|
##### `CommandRun`
|
||||||
|
- `command: String` — Original command display text.
|
||||||
|
- `lines: Flow<String>` — Streaming stdout lines. Use for large output.
|
||||||
|
- `errorLines: Flow<String>` — Streaming stderr lines. Use for large error output.
|
||||||
|
- `out: String` — Captured stdout. Captures both stdout and stderr concurrently and waits for completion.
|
||||||
|
- `err: String` — Captured stderr. Captures both stdout and stderr concurrently and waits for completion.
|
||||||
|
- `wait(): Int` — Wait for the command to exit and return the exit code.
|
||||||
|
- `check(): CommandRun` — Capture output, wait, and fail if the exit code is non-zero.
|
||||||
|
- `code: Int?` — Known exit code after `wait`, `check`, `out`, or `err`; otherwise `null`.
|
||||||
|
- `ok: Bool?` — `true` for exit code 0, `false` for non-zero, or `null` before the exit code is known.
|
||||||
|
|
||||||
##### `RunningProcess` (instance methods)
|
##### `RunningProcess` (instance methods)
|
||||||
- `stdout: Flow` — Standard output stream as a Lyng Flow of lines.
|
- `stdout(): Flow` — Standard output stream as a Lyng Flow of lines.
|
||||||
- `stderr: Flow` — Standard error stream as a Lyng Flow of lines.
|
- `stderr(): Flow` — Standard error stream as a Lyng Flow of lines.
|
||||||
- `waitFor(): Int` — Wait for the process to exit and return the exit code.
|
- `waitFor(): Int` — Wait for the process to exit and return the exit code.
|
||||||
- `signal(name: String)` — Send a signal to the process (e.g., `"SIGINT"`, `"SIGTERM"`, `"SIGKILL"`).
|
- `signal(name: String)` — Send a signal to the process (e.g., `"SIGINT"`, `"SIGTERM"`, `"SIGKILL"`).
|
||||||
- `destroy()` — Forcefully terminate the process.
|
- `destroy()` — Forcefully terminate the process.
|
||||||
@ -110,6 +370,7 @@ Example of a restricted policy in Kotlin:
|
|||||||
|
|
||||||
```kotlin
|
```kotlin
|
||||||
import net.sergeych.lyngio.fs.security.AccessDecision
|
import net.sergeych.lyngio.fs.security.AccessDecision
|
||||||
|
import net.sergeych.lyngio.fs.security.AccessContext
|
||||||
import net.sergeych.lyngio.fs.security.Decision
|
import net.sergeych.lyngio.fs.security.Decision
|
||||||
import net.sergeych.lyngio.process.security.ProcessAccessOp
|
import net.sergeych.lyngio.process.security.ProcessAccessOp
|
||||||
import net.sergeych.lyngio.process.security.ProcessAccessPolicy
|
import net.sergeych.lyngio.process.security.ProcessAccessPolicy
|
||||||
|
|||||||
@ -4,6 +4,7 @@ The Lyng CLI is the reference command-line tool for the Lyng language. It lets y
|
|||||||
|
|
||||||
- Run Lyng scripts from files or inline strings (shebangs accepted)
|
- Run Lyng scripts from files or inline strings (shebangs accepted)
|
||||||
- Use standard argument passing (`ARGV`) to your scripts.
|
- Use standard argument passing (`ARGV`) to your scripts.
|
||||||
|
- Import `lyng.io.process` for shell-script style process execution (`sh`, `exec`, and `CommandRun`).
|
||||||
- Resolve local file imports from the executed script's directory tree.
|
- Resolve local file imports from the executed script's directory tree.
|
||||||
- Format Lyng source files via the built-in `fmt` subcommand.
|
- Format Lyng source files via the built-in `fmt` subcommand.
|
||||||
- Register synchronous process-exit handlers with `atExit(...)`.
|
- Register synchronous process-exit handlers with `atExit(...)`.
|
||||||
@ -81,6 +82,15 @@ lyng -- -my-script.lyng arg1 arg2
|
|||||||
lyng -x "println(\"Hello\")" more args
|
lyng -x "println(\"Hello\")" more args
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The CLI installs `lyng.io.process`, so scripts can use the shell/process shorthand after importing it:
|
||||||
|
|
||||||
|
```lyng
|
||||||
|
import lyng.io.process
|
||||||
|
|
||||||
|
val branch = sh("git branch --show-current").out.trim()
|
||||||
|
exec("git", ["status", "--short"]).check()
|
||||||
|
```
|
||||||
|
|
||||||
- Print version/help:
|
- Print version/help:
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|||||||
@ -14,7 +14,7 @@
|
|||||||
|
|
||||||
- **[lyng.io.db](lyng.io.db.md):** Portable SQL database access. Provides `Database`, `SqlTransaction`, `ResultSet`, SQLite support through `lyng.io.db.sqlite`, and JVM JDBC support through `lyng.io.db.jdbc`.
|
- **[lyng.io.db](lyng.io.db.md):** Portable SQL database access. Provides `Database`, `SqlTransaction`, `ResultSet`, SQLite support through `lyng.io.db.sqlite`, and JVM JDBC support through `lyng.io.db.jdbc`.
|
||||||
- **[lyng.io.fs](lyng.io.fs.md):** Async filesystem access. Provides the `Path` class for file/directory operations, streaming, and globbing.
|
- **[lyng.io.fs](lyng.io.fs.md):** Async filesystem access. Provides the `Path` class for file/directory operations, streaming, and globbing.
|
||||||
- **[lyng.io.process](lyng.io.process.md):** External process execution and shell commands. Provides `Process`, `RunningProcess`, and `Platform` information.
|
- **[lyng.io.process](lyng.io.process.md):** External process execution and shell commands. Provides shell-script sugar (`sh`, `exec`, `CommandRun`), low-level `Process`/`RunningProcess`, and `Platform` information.
|
||||||
- **[lyng.io.console](lyng.io.console.md):** Rich console/TTY access. Provides `Console` capability detection, geometry, output, and iterable events.
|
- **[lyng.io.console](lyng.io.console.md):** Rich console/TTY access. Provides `Console` capability detection, geometry, output, and iterable events.
|
||||||
- **[lyng.io.http](lyng.io.http.md):** HTTP/HTTPS client access. Provides `Http`, `HttpRequest`, `HttpResponse`, and `HttpHeaders`.
|
- **[lyng.io.http](lyng.io.http.md):** HTTP/HTTPS client access. Provides `Http`, `HttpRequest`, `HttpResponse`, and `HttpHeaders`.
|
||||||
- **[lyng.io.http.server](lyng.io.http.server.md):** Minimal HTTP/1.1 and WebSocket server. Provides `HttpServer`, `Router`, `ServerRequest`, `RequestContext`, and `ServerWebSocket`.
|
- **[lyng.io.http.server](lyng.io.http.server.md):** Minimal HTTP/1.1 and WebSocket server. Provides `HttpServer`, `Router`, `ServerRequest`, `RequestContext`, and `ServerWebSocket`.
|
||||||
|
|||||||
@ -49,6 +49,7 @@ import net.sergeych.lyng.io.html.createHtmlModule
|
|||||||
import net.sergeych.lyng.io.http.createHttpModule
|
import net.sergeych.lyng.io.http.createHttpModule
|
||||||
import net.sergeych.lyng.io.http.server.createHttpServerModule
|
import net.sergeych.lyng.io.http.server.createHttpServerModule
|
||||||
import net.sergeych.lyng.io.net.createNetModule
|
import net.sergeych.lyng.io.net.createNetModule
|
||||||
|
import net.sergeych.lyng.io.process.createProcessModule
|
||||||
import net.sergeych.lyng.io.ws.createWsModule
|
import net.sergeych.lyng.io.ws.createWsModule
|
||||||
import net.sergeych.lyng.obj.*
|
import net.sergeych.lyng.obj.*
|
||||||
import net.sergeych.lyng.pacman.ImportManager
|
import net.sergeych.lyng.pacman.ImportManager
|
||||||
@ -57,6 +58,7 @@ import net.sergeych.lyngio.fs.security.PermitAllAccessPolicy
|
|||||||
import net.sergeych.lyngio.http.security.PermitAllHttpAccessPolicy
|
import net.sergeych.lyngio.http.security.PermitAllHttpAccessPolicy
|
||||||
import net.sergeych.lyngio.net.security.PermitAllNetAccessPolicy
|
import net.sergeych.lyngio.net.security.PermitAllNetAccessPolicy
|
||||||
import net.sergeych.lyngio.net.shutdownSystemNetEngine
|
import net.sergeych.lyngio.net.shutdownSystemNetEngine
|
||||||
|
import net.sergeych.lyngio.process.security.PermitAllProcessAccessPolicy
|
||||||
import net.sergeych.lyngio.ws.security.PermitAllWsAccessPolicy
|
import net.sergeych.lyngio.ws.security.PermitAllWsAccessPolicy
|
||||||
import net.sergeych.mp_tools.globalDefer
|
import net.sergeych.mp_tools.globalDefer
|
||||||
import okio.*
|
import okio.*
|
||||||
@ -150,6 +152,7 @@ private fun ImportManager.invalidateCliModuleCaches() {
|
|||||||
invalidatePackageCache("lyng.io.html")
|
invalidatePackageCache("lyng.io.html")
|
||||||
invalidatePackageCache("lyng.io.http")
|
invalidatePackageCache("lyng.io.http")
|
||||||
invalidatePackageCache("lyng.io.http.server")
|
invalidatePackageCache("lyng.io.http.server")
|
||||||
|
invalidatePackageCache("lyng.io.process")
|
||||||
invalidatePackageCache("lyng.io.ws")
|
invalidatePackageCache("lyng.io.ws")
|
||||||
invalidatePackageCache("lyng.io.net")
|
invalidatePackageCache("lyng.io.net")
|
||||||
}
|
}
|
||||||
@ -242,6 +245,7 @@ private fun installCliModules(manager: ImportManager) {
|
|||||||
createHtmlModule(manager)
|
createHtmlModule(manager)
|
||||||
createHttpModule(PermitAllHttpAccessPolicy, manager)
|
createHttpModule(PermitAllHttpAccessPolicy, manager)
|
||||||
createHttpServerModule(PermitAllNetAccessPolicy, manager)
|
createHttpServerModule(PermitAllNetAccessPolicy, manager)
|
||||||
|
createProcessModule(PermitAllProcessAccessPolicy, manager)
|
||||||
createWsModule(PermitAllWsAccessPolicy, manager)
|
createWsModule(PermitAllWsAccessPolicy, manager)
|
||||||
createNetModule(PermitAllNetAccessPolicy, manager)
|
createNetModule(PermitAllNetAccessPolicy, manager)
|
||||||
}
|
}
|
||||||
|
|||||||
@ -17,10 +17,16 @@
|
|||||||
|
|
||||||
package net.sergeych.lyng.io.process
|
package net.sergeych.lyng.io.process
|
||||||
|
|
||||||
|
import kotlinx.coroutines.async
|
||||||
|
import kotlinx.coroutines.coroutineScope
|
||||||
import kotlinx.coroutines.flow.Flow
|
import kotlinx.coroutines.flow.Flow
|
||||||
|
import kotlinx.coroutines.flow.toList
|
||||||
|
import kotlinx.coroutines.sync.Mutex
|
||||||
|
import kotlinx.coroutines.sync.withLock
|
||||||
import net.sergeych.lyng.ModuleScope
|
import net.sergeych.lyng.ModuleScope
|
||||||
import net.sergeych.lyng.Scope
|
import net.sergeych.lyng.Scope
|
||||||
import net.sergeych.lyng.ScopeFacade
|
import net.sergeych.lyng.ScopeFacade
|
||||||
|
import net.sergeych.lyng.Source
|
||||||
import net.sergeych.lyng.miniast.*
|
import net.sergeych.lyng.miniast.*
|
||||||
import net.sergeych.lyng.obj.*
|
import net.sergeych.lyng.obj.*
|
||||||
import net.sergeych.lyng.pacman.ImportManager
|
import net.sergeych.lyng.pacman.ImportManager
|
||||||
@ -28,6 +34,24 @@ import net.sergeych.lyng.requireScope
|
|||||||
import net.sergeych.lyngio.process.*
|
import net.sergeych.lyngio.process.*
|
||||||
import net.sergeych.lyngio.process.security.ProcessAccessDeniedException
|
import net.sergeych.lyngio.process.security.ProcessAccessDeniedException
|
||||||
import net.sergeych.lyngio.process.security.ProcessAccessPolicy
|
import net.sergeych.lyngio.process.security.ProcessAccessPolicy
|
||||||
|
import net.sergeych.lyngio.stdlib_included.processLyng
|
||||||
|
import kotlin.Boolean
|
||||||
|
import kotlin.Exception
|
||||||
|
import kotlin.Int
|
||||||
|
import kotlin.String
|
||||||
|
import kotlin.also
|
||||||
|
import kotlin.apply
|
||||||
|
import kotlin.collections.joinToString
|
||||||
|
import kotlin.collections.listOf
|
||||||
|
import kotlin.collections.map
|
||||||
|
import kotlin.collections.mutableListOf
|
||||||
|
import kotlin.collections.mutableMapOf
|
||||||
|
import kotlin.collections.toTypedArray
|
||||||
|
import kotlin.let
|
||||||
|
import kotlin.takeIf
|
||||||
|
import kotlin.text.isNotBlank
|
||||||
|
import kotlin.text.uppercase
|
||||||
|
import kotlin.to
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Install Lyng module `lyng.io.process` into the given scope's ImportManager.
|
* Install Lyng module `lyng.io.process` into the given scope's ImportManager.
|
||||||
@ -47,6 +71,8 @@ fun createProcessModule(policy: ProcessAccessPolicy, manager: ImportManager): Bo
|
|||||||
}
|
}
|
||||||
|
|
||||||
private suspend fun buildProcessModule(module: ModuleScope, policy: ProcessAccessPolicy) {
|
private suspend fun buildProcessModule(module: ModuleScope, policy: ProcessAccessPolicy) {
|
||||||
|
module.eval(Source("lyng.io.process", processLyng))
|
||||||
|
|
||||||
val runner = try {
|
val runner = try {
|
||||||
SecuredLyngProcessRunner(getSystemProcessRunner(), policy)
|
SecuredLyngProcessRunner(getSystemProcessRunner(), policy)
|
||||||
} catch (e: Exception) {
|
} catch (e: Exception) {
|
||||||
@ -54,6 +80,7 @@ private suspend fun buildProcessModule(module: ModuleScope, policy: ProcessAcces
|
|||||||
}
|
}
|
||||||
|
|
||||||
val runningProcessType = object : ObjClass("RunningProcess") {}
|
val runningProcessType = object : ObjClass("RunningProcess") {}
|
||||||
|
val commandRunType = object : ObjClass("CommandRun") {}
|
||||||
|
|
||||||
runningProcessType.apply {
|
runningProcessType.apply {
|
||||||
addFnDoc(
|
addFnDoc(
|
||||||
@ -149,6 +176,87 @@ private suspend fun buildProcessModule(module: ModuleScope, policy: ProcessAcces
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
commandRunType.apply {
|
||||||
|
addPropertyDoc(
|
||||||
|
name = "command",
|
||||||
|
doc = "Original shell command or argv-style command display text.",
|
||||||
|
type = type("lyng.String"),
|
||||||
|
moduleName = module.packageName,
|
||||||
|
getter = { ObjString(thisAs<ObjCommandRun>().command) }
|
||||||
|
)
|
||||||
|
addPropertyDoc(
|
||||||
|
name = "out",
|
||||||
|
doc = "Captured standard output as a string. Captures both stdout and stderr concurrently.",
|
||||||
|
type = type("lyng.String"),
|
||||||
|
moduleName = module.packageName,
|
||||||
|
getter = { ObjString(thisAs<ObjCommandRun>().captureAll().stdoutText) }
|
||||||
|
)
|
||||||
|
addPropertyDoc(
|
||||||
|
name = "err",
|
||||||
|
doc = "Captured standard error as a string. Captures both stdout and stderr concurrently.",
|
||||||
|
type = type("lyng.String"),
|
||||||
|
moduleName = module.packageName,
|
||||||
|
getter = { ObjString(thisAs<ObjCommandRun>().captureAll().stderrText) }
|
||||||
|
)
|
||||||
|
addPropertyDoc(
|
||||||
|
name = "lines",
|
||||||
|
doc = "Streaming standard output lines. Use this for large output instead of `out`.",
|
||||||
|
type = type("lyng.Flow"),
|
||||||
|
moduleName = module.packageName,
|
||||||
|
getter = { thisAs<ObjCommandRun>().process.stdout.toLyngFlow(this) }
|
||||||
|
)
|
||||||
|
addPropertyDoc(
|
||||||
|
name = "errorLines",
|
||||||
|
doc = "Streaming standard error lines. Use this for large stderr output instead of `err`.",
|
||||||
|
type = type("lyng.Flow"),
|
||||||
|
moduleName = module.packageName,
|
||||||
|
getter = { thisAs<ObjCommandRun>().process.stderr.toLyngFlow(this) }
|
||||||
|
)
|
||||||
|
addPropertyDoc(
|
||||||
|
name = "code",
|
||||||
|
doc = "Exit code after `wait`, `check`, `out`, or `err`; otherwise null.",
|
||||||
|
type = type("lyng.Int?"),
|
||||||
|
moduleName = module.packageName,
|
||||||
|
getter = {
|
||||||
|
val code = thisAs<ObjCommandRun>().knownExitCode()
|
||||||
|
code?.toObj() ?: ObjNull
|
||||||
|
}
|
||||||
|
)
|
||||||
|
addPropertyDoc(
|
||||||
|
name = "ok",
|
||||||
|
doc = "True if the known exit code is zero, false if non-zero, or null before the process is known to have exited.",
|
||||||
|
type = type("lyng.Bool?"),
|
||||||
|
moduleName = module.packageName,
|
||||||
|
getter = {
|
||||||
|
val code = thisAs<ObjCommandRun>().knownExitCode()
|
||||||
|
code?.let { (it == 0).toObj() } ?: ObjNull
|
||||||
|
}
|
||||||
|
)
|
||||||
|
addFnDoc(
|
||||||
|
name = "wait",
|
||||||
|
doc = "Wait for the process to exit and return its exit code.",
|
||||||
|
returns = type("lyng.Int"),
|
||||||
|
moduleName = module.packageName
|
||||||
|
) {
|
||||||
|
thisAs<ObjCommandRun>().waitFor().toObj()
|
||||||
|
}
|
||||||
|
addFnDoc(
|
||||||
|
name = "check",
|
||||||
|
doc = "Capture output, wait for completion, and fail if the exit code is non-zero.",
|
||||||
|
returns = type("CommandRun"),
|
||||||
|
moduleName = module.packageName
|
||||||
|
) {
|
||||||
|
val command = thisAs<ObjCommandRun>()
|
||||||
|
val captured = command.captureAll()
|
||||||
|
if (captured.exitCode != 0) {
|
||||||
|
val detail = captured.stderrText.takeIf { it.isNotBlank() } ?: captured.stdoutText
|
||||||
|
val suffix = detail.takeIf { it.isNotBlank() }?.let { ": $it" } ?: ""
|
||||||
|
raiseError("command failed with exit code ${captured.exitCode}: ${command.command}$suffix")
|
||||||
|
}
|
||||||
|
command
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
val platformType = object : ObjClass("Platform") {}
|
val platformType = object : ObjClass("Platform") {}
|
||||||
|
|
||||||
platformType.apply {
|
platformType.apply {
|
||||||
@ -197,6 +305,45 @@ private suspend fun buildProcessModule(module: ModuleScope, policy: ProcessAcces
|
|||||||
type = type("RunningProcess"),
|
type = type("RunningProcess"),
|
||||||
moduleName = module.packageName
|
moduleName = module.packageName
|
||||||
)
|
)
|
||||||
|
module.addConstDoc(
|
||||||
|
name = "CommandRun",
|
||||||
|
value = commandRunType,
|
||||||
|
doc = "Shell-script friendly handle for a running command.",
|
||||||
|
type = type("CommandRun"),
|
||||||
|
moduleName = module.packageName
|
||||||
|
)
|
||||||
|
|
||||||
|
module.addFnDoc(
|
||||||
|
"sh",
|
||||||
|
doc = "Run a command via the system shell and return an active command handle.",
|
||||||
|
params = listOf(ParamDoc("command", type("lyng.String"))),
|
||||||
|
returns = type("CommandRun"),
|
||||||
|
moduleName = module.packageName
|
||||||
|
) {
|
||||||
|
if (runner == null) raiseError("Processes are not supported on this platform")
|
||||||
|
processGuard {
|
||||||
|
val command = requireOnlyArg<ObjString>().value
|
||||||
|
ObjCommandRun(commandRunType, command, runner.shell(command))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
module.addFnDoc(
|
||||||
|
"exec",
|
||||||
|
doc = "Run an executable with argv-style arguments and return an active command handle.",
|
||||||
|
params = listOf(ParamDoc("executable", type("lyng.String")), ParamDoc("args", type("lyng.List"))),
|
||||||
|
returns = type("CommandRun"),
|
||||||
|
moduleName = module.packageName
|
||||||
|
) {
|
||||||
|
if (runner == null) raiseError("Processes are not supported on this platform")
|
||||||
|
processGuard {
|
||||||
|
if (args.list.size > 2) {
|
||||||
|
raiseError("Expected at most 2 arguments, got ${args.list.size}")
|
||||||
|
}
|
||||||
|
val executable = requiredArg<ObjString>(0).value
|
||||||
|
val rawArgs = if (args.list.size >= 2) requiredArg<ObjList>(1) else ObjList(mutableListOf())
|
||||||
|
val argv = rawArgs.list.map { (it as? ObjString)?.value ?: it.toString() }
|
||||||
|
ObjCommandRun(commandRunType, listOf(executable, *argv.toTypedArray()).joinToString(" "), runner.execute(executable, argv))
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
class ObjRunningProcess(
|
class ObjRunningProcess(
|
||||||
@ -206,6 +353,49 @@ class ObjRunningProcess(
|
|||||||
override fun toString(): String = "RunningProcess($process)"
|
override fun toString(): String = "RunningProcess($process)"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private data class CapturedCommandOutput(
|
||||||
|
val exitCode: Int,
|
||||||
|
val stdoutText: String,
|
||||||
|
val stderrText: String
|
||||||
|
)
|
||||||
|
|
||||||
|
private class ObjCommandRun(
|
||||||
|
override val objClass: ObjClass,
|
||||||
|
val command: String,
|
||||||
|
val process: LyngProcess
|
||||||
|
) : Obj() {
|
||||||
|
private val captureMutex = Mutex()
|
||||||
|
private var captured: CapturedCommandOutput? = null
|
||||||
|
private var exitCode: Int? = null
|
||||||
|
|
||||||
|
fun knownExitCode(): Int? = captured?.exitCode ?: exitCode
|
||||||
|
|
||||||
|
suspend fun waitFor(): Int {
|
||||||
|
captured?.let { return it.exitCode }
|
||||||
|
exitCode?.let { return it }
|
||||||
|
return process.waitFor().also { exitCode = it }
|
||||||
|
}
|
||||||
|
|
||||||
|
suspend fun captureAll(): CapturedCommandOutput = captureMutex.withLock {
|
||||||
|
captured?.let { return@withLock it }
|
||||||
|
coroutineScope {
|
||||||
|
val stdout = async { process.stdout.toList().joinToString("\n") }
|
||||||
|
val stderr = async { process.stderr.toList().joinToString("\n") }
|
||||||
|
val code = async { process.waitFor() }
|
||||||
|
CapturedCommandOutput(
|
||||||
|
exitCode = code.await(),
|
||||||
|
stdoutText = stdout.await(),
|
||||||
|
stderrText = stderr.await()
|
||||||
|
).also {
|
||||||
|
captured = it
|
||||||
|
exitCode = it.exitCode
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
override fun toString(): String = "CommandRun($command)"
|
||||||
|
}
|
||||||
|
|
||||||
private suspend inline fun ScopeFacade.processGuard(crossinline block: suspend () -> Obj): Obj {
|
private suspend inline fun ScopeFacade.processGuard(crossinline block: suspend () -> Obj): Obj {
|
||||||
return try {
|
return try {
|
||||||
block()
|
block()
|
||||||
|
|||||||
@ -19,9 +19,11 @@ package net.sergeych.lyng.io.process
|
|||||||
|
|
||||||
import kotlinx.coroutines.runBlocking
|
import kotlinx.coroutines.runBlocking
|
||||||
import net.sergeych.lyng.Compiler
|
import net.sergeych.lyng.Compiler
|
||||||
|
import net.sergeych.lyng.ExecutionError
|
||||||
import net.sergeych.lyng.Script
|
import net.sergeych.lyng.Script
|
||||||
import net.sergeych.lyngio.process.security.PermitAllProcessAccessPolicy
|
import net.sergeych.lyngio.process.security.PermitAllProcessAccessPolicy
|
||||||
import kotlin.test.Test
|
import kotlin.test.Test
|
||||||
|
import kotlin.test.assertFailsWith
|
||||||
import kotlin.test.assertTrue
|
import kotlin.test.assertTrue
|
||||||
|
|
||||||
class LyngProcessModuleTest {
|
class LyngProcessModuleTest {
|
||||||
@ -84,4 +86,78 @@ class LyngProcessModuleTest {
|
|||||||
val result = script.execute(scope)
|
val result = script.execute(scope)
|
||||||
assertTrue(result.inspect(scope).contains("name"), "Result should contain 'name', but was: ${result.inspect(scope)}")
|
assertTrue(result.inspect(scope).contains("name"), "Result should contain 'name', but was: ${result.inspect(scope)}")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun testShellSugarCapturesOutput() = runBlocking {
|
||||||
|
val scope = Script.newScope()
|
||||||
|
createProcessModule(PermitAllProcessAccessPolicy, scope)
|
||||||
|
|
||||||
|
val code = """
|
||||||
|
import lyng.io.process
|
||||||
|
|
||||||
|
val r = sh("echo shell-sugar")
|
||||||
|
assert(r.out.trim() == "shell-sugar")
|
||||||
|
assert(r.code == 0)
|
||||||
|
assert(r.ok == true)
|
||||||
|
true
|
||||||
|
""".trimIndent()
|
||||||
|
|
||||||
|
val script = Compiler.compile(code)
|
||||||
|
val result = script.execute(scope)
|
||||||
|
assertTrue(result.inspect(scope).contains("true"))
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun testShellSugarStreamsLines() = runBlocking {
|
||||||
|
val scope = Script.newScope()
|
||||||
|
createProcessModule(PermitAllProcessAccessPolicy, scope)
|
||||||
|
|
||||||
|
val code = """
|
||||||
|
import lyng.io.process
|
||||||
|
|
||||||
|
val lines: List<String> = []
|
||||||
|
for (line in sh("echo one && echo two").lines) {
|
||||||
|
lines.add(line)
|
||||||
|
}
|
||||||
|
lines.joinToString(",")
|
||||||
|
""".trimIndent()
|
||||||
|
|
||||||
|
val script = Compiler.compile(code)
|
||||||
|
val result = script.execute(scope)
|
||||||
|
assertTrue(result.inspect(scope).contains("one,two"))
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun testExecSugarCapturesOutput() = runBlocking {
|
||||||
|
val scope = Script.newScope()
|
||||||
|
createProcessModule(PermitAllProcessAccessPolicy, scope)
|
||||||
|
|
||||||
|
val code = """
|
||||||
|
import lyng.io.process
|
||||||
|
|
||||||
|
exec("echo", ["exec-sugar"]).out.trim()
|
||||||
|
""".trimIndent()
|
||||||
|
|
||||||
|
val script = Compiler.compile(code)
|
||||||
|
val result = script.execute(scope)
|
||||||
|
assertTrue(result.inspect(scope).contains("exec-sugar"))
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun testShellSugarCheckFailsOnNonZeroExit() = runBlocking {
|
||||||
|
val scope = Script.newScope()
|
||||||
|
createProcessModule(PermitAllProcessAccessPolicy, scope)
|
||||||
|
|
||||||
|
val code = """
|
||||||
|
import lyng.io.process
|
||||||
|
|
||||||
|
sh("echo check-failed && exit 7").check()
|
||||||
|
""".trimIndent()
|
||||||
|
|
||||||
|
val script = Compiler.compile(code)
|
||||||
|
assertFailsWith<ExecutionError> {
|
||||||
|
script.execute(scope)
|
||||||
|
}
|
||||||
|
Unit
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
59
lyngio/stdlib/lyng/io/process.lyng
Normal file
59
lyngio/stdlib/lyng/io/process.lyng
Normal file
@ -0,0 +1,59 @@
|
|||||||
|
package lyng.io.process
|
||||||
|
|
||||||
|
/* Handle to a running process. This is the low-level process API. */
|
||||||
|
extern class RunningProcess {
|
||||||
|
/* Standard output as a flow of process output lines. */
|
||||||
|
fun stdout(): Flow<String>
|
||||||
|
/* Standard error as a flow of process error lines. */
|
||||||
|
fun stderr(): Flow<String>
|
||||||
|
/* Send a signal such as "SIGINT", "SIGTERM", or "SIGKILL". */
|
||||||
|
fun signal(signal: String): void
|
||||||
|
/* Wait for process completion and return its exit code. */
|
||||||
|
fun waitFor(): Int
|
||||||
|
/* Forcefully terminate the process. */
|
||||||
|
fun destroy(): void
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Shell-script friendly active handle for a running command. */
|
||||||
|
extern class CommandRun {
|
||||||
|
/* Original shell command or argv-style command display text. */
|
||||||
|
val command: String
|
||||||
|
/* Capture standard output as a string, wait for completion, and drain stderr concurrently. Use `lines` for large output. */
|
||||||
|
val out: String
|
||||||
|
/* Capture standard error as a string, wait for completion, and drain stdout concurrently. Use `errorLines` for large output. */
|
||||||
|
val err: String
|
||||||
|
/* Streaming standard output lines. Treat the process stdout stream as one-shot. */
|
||||||
|
val lines: Flow<String>
|
||||||
|
/* Streaming standard error lines. Treat the process stderr stream as one-shot. */
|
||||||
|
val errorLines: Flow<String>
|
||||||
|
/* Exit code after `wait`, `check`, `out`, or `err`; otherwise null. */
|
||||||
|
val code: Int?
|
||||||
|
/* True for exit code 0, false for non-zero, or null before the exit code is known. */
|
||||||
|
val ok: Bool?
|
||||||
|
/* Wait for the command to exit and return the exit code. */
|
||||||
|
fun wait(): Int
|
||||||
|
/* Capture output, wait for completion, and fail if the exit code is non-zero. */
|
||||||
|
fun check(): CommandRun
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Process execution and control. */
|
||||||
|
extern object Process {
|
||||||
|
/* Execute a process with arguments without using a shell. */
|
||||||
|
fun execute(executable: String, args: List<String>): RunningProcess
|
||||||
|
/* Execute a command via the system shell. */
|
||||||
|
fun shell(command: String): RunningProcess
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Platform information for process support. */
|
||||||
|
extern object Platform {
|
||||||
|
/* Get platform core details. */
|
||||||
|
fun details(): Map<String,Object?>
|
||||||
|
/* Check if processes are supported on this platform. */
|
||||||
|
fun isSupported(): Bool
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Run a command via the system shell and return an active command handle. */
|
||||||
|
extern fun sh(command: String): CommandRun
|
||||||
|
|
||||||
|
/* Run an executable with argv-style arguments and return an active command handle. */
|
||||||
|
extern fun exec(executable: String, args: List<String> = []): CommandRun
|
||||||
Loading…
x
Reference in New Issue
Block a user