Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

The "url" paramater is not working in the embulk-output-sqlserver plugin #349

Open
kei-hito opened this issue Sep 13, 2024 · 0 comments
Open

The "url" paramater is not working in the embulk-output-sqlserver plugin #349

kei-hito opened this issue Sep 13, 2024 · 0 comments

Comments

@kei-hito
Copy link

kei-hito commented Sep 13, 2024 •
edited
Loading

ENV

  • OS: Debian 11.4
  • Java: openjdk version "1.8.0_342
  • embulk-0.9.24
  • embulk-output-sqlserver: 0.10.2

TO REPRODUCE

I ran the following config file and got an error.

[config.yml]

in:
  type: file
  path_prefix: ****.csv

out:
  type: sqlserver
  url: jdbc:sqlserver://xxxxxxx:1433
  user: xxxxxxx
  password: xxxxxxxx
  database: xxxxx
  table: sample_table
  mode: insert_direct

[Error Message]

2024-09-10 08:21:50.815 +0000 [INFO] (0069@[0:p_5dc01c89-8aac-44e3-b2c1-2871ff90392e_1]+w_1+task0) io.digdag.core.agent.OperatorManager: sh>: embulk run config.yml.
2024-09-10 08:21:51.145 +0000: Embulk v0.9.24
2024-09-10 08:21:51.972 +0000 [WARN] (main): DEPRECATION: JRuby org.jruby.embed.ScriptingContainer is directly injected.
2024-09-10 08:21:54.057 +0000 [INFO] (main): Gem's home and path are set by default: "/root/.embulk/lib/gems"
2024-09-10 08:21:55.795 +0000 [INFO] (main): Started Embulk v0.9.24
2024-09-10 08:21:55.901 +0000 [INFO] (0001:transaction): Loaded plugin embulk-output-sqlserver (0.10.2)
2024-09-10 08:21:55.960 +0000 [INFO] (0001:transaction): Listing local files at directory '/opt/xxxxxxx' filtering filename by prefix 'xxxxxx.csv'
2024-09-10 08:21:55.961 +0000 [INFO] (0001:transaction): "follow_symlinks" is set false. Note that symbolic links to directories are skipped.
2024-09-10 08:21:56.080 +0000 [INFO] (0001:transaction): Loading files [/opt/xxxxxxxxx.csv]
2024-09-10 08:21:56.108 +0000 [INFO] (0001:transaction): Using local thread executor with max_threads=16 / output tasks 8 = input tasks 1 * 8
2024-09-10 08:21:56.135 +0000 [WARN] (0001:transaction): "UTC" is recognized as "Z" to be compatible with the legacy style.
org.embulk.exec.PartialExecutionException: java.util.NoSuchElementException: No value present
        at org.embulk.exec.BulkLoader$LoaderState.buildPartialExecuteException(BulkLoader.java:340)
        at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:566)
        at org.embulk.exec.BulkLoader.access$000(BulkLoader.java:35)
        at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:353)
        at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:350)
        at org.embulk.spi.Exec.doWith(Exec.java:23)
        at org.embulk.exec.BulkLoader.run(BulkLoader.java:350)
        at org.embulk.EmbulkEmbed.run(EmbulkEmbed.java:242)
        at org.embulk.EmbulkRunner.runInternal(EmbulkRunner.java:291)
        at org.embulk.EmbulkRunner.run(EmbulkRunner.java:155)
        at org.embulk.cli.EmbulkRun.runSubcommand(EmbulkRun.java:431)
        at org.embulk.cli.EmbulkRun.run(EmbulkRun.java:90)
        at org.embulk.cli.Main.main(Main.java:64)
        Suppressed: java.lang.NullPointerException
                at org.embulk.exec.BulkLoader.doCleanup(BulkLoader.java:463)
                at org.embulk.exec.BulkLoader$3.run(BulkLoader.java:397)
                at org.embulk.exec.BulkLoader$3.run(BulkLoader.java:394)
                at org.embulk.spi.Exec.doWith(Exec.java:23)
                at org.embulk.exec.BulkLoader.cleanup(BulkLoader.java:394)
                at org.embulk.EmbulkEmbed.run(EmbulkEmbed.java:245)
                ... 5 more
Caused by: java.util.NoSuchElementException: No value present
        at java.util.Optional.get(Optional.java:135)
        at org.embulk.output.SQLServerOutputPlugin.getUrlAndProperties(SQLServerOutputPlugin.java:252)
        at org.embulk.output.SQLServerOutputPlugin.getConnector(SQLServerOutputPlugin.java:175)
        at org.embulk.output.jdbc.AbstractJdbcOutputPlugin.newConnection(AbstractJdbcOutputPlugin.java:352)
        at org.embulk.output.jdbc.AbstractJdbcOutputPlugin$2.run(AbstractJdbcOutputPlugin.java:488)
        at org.embulk.output.jdbc.AbstractJdbcOutputPlugin$RetryableSQLExecution.call(AbstractJdbcOutputPlugin.java:1343)
        at org.embulk.output.jdbc.AbstractJdbcOutputPlugin$RetryableSQLExecution.call(AbstractJdbcOutputPlugin.java:1331)
        at org.embulk.util.retryhelper.RetryExecutor.run(RetryExecutor.java:109)
        at org.embulk.util.retryhelper.RetryExecutor.runInterruptible(RetryExecutor.java:90)
        at org.embulk.output.jdbc.AbstractJdbcOutputPlugin.withRetry(AbstractJdbcOutputPlugin.java:1309)
        at org.embulk.output.jdbc.AbstractJdbcOutputPlugin.withRetry(AbstractJdbcOutputPlugin.java:1301)
        at org.embulk.output.jdbc.AbstractJdbcOutputPlugin.begin(AbstractJdbcOutputPlugin.java:485)
        at org.embulk.output.jdbc.AbstractJdbcOutputPlugin.transaction(AbstractJdbcOutputPlugin.java:461)
        at org.embulk.exec.BulkLoader$4$1$1.transaction(BulkLoader.java:521)
        at org.embulk.exec.LocalExecutorPlugin.transaction(LocalExecutorPlugin.java:50)
        at org.embulk.exec.BulkLoader$4$1.run(BulkLoader.java:516)
        at org.embulk.spi.util.Filters$RecursiveControl.transaction(Filters.java:84)
        at org.embulk.spi.util.Filters.transaction(Filters.java:42)
        at org.embulk.exec.BulkLoader$4.run(BulkLoader.java:511)
        at org.embulk.spi.FileInputRunner$RunnerControl$1$1.run(FileInputRunner.java:112)
        at org.embulk.standards.CsvParserPlugin.transaction(CsvParserPlugin.java:237)
        at org.embulk.spi.FileInputRunner$RunnerControl$1.run(FileInputRunner.java:107)
        at org.embulk.spi.util.Decoders$RecursiveControl.transaction(Decoders.java:68)
        at org.embulk.spi.util.Decoders.transaction(Decoders.java:29)
        at org.embulk.spi.FileInputRunner$RunnerControl.run(FileInputRunner.java:105)
        at org.embulk.standards.LocalFileInputPlugin.resume(LocalFileInputPlugin.java:79)
        at org.embulk.standards.LocalFileInputPlugin.transaction(LocalFileInputPlugin.java:72)
        at org.embulk.spi.FileInputRunner.transaction(FileInputRunner.java:62)
        at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:507)
        ... 11 more

Error: java.util.NoSuchElementException: No value present
2024-09-10 08:21:56.173 +0000 [ERROR] (0069@[0:p_5dc01c89-8aac-44e3-b2c1-2871ff90392e_1]+w_1+task-File-to-AzureSQLDB-0) io.digdag.core.agent.OperatorManager: Task failed with unexpected error: Command failed with code 1
java.lang.RuntimeException: Command failed with code 1
        at io.digdag.standards.operator.ShOperatorFactory$ShOperator.runCode(ShOperatorFactory.java:121)
        at io.digdag.standards.operator.ShOperatorFactory$ShOperator.runTask(ShOperatorFactory.java:88)
        at io.digdag.util.BaseOperator.run(BaseOperator.java:35)
        at io.digdag.core.agent.OperatorManager.callExecutor(OperatorManager.java:399)
        at io.digdag.server.metrics.DigdagTimedMethodInterceptor.invokeMain(DigdagTimedMethodInterceptor.java:58)
        at io.digdag.server.metrics.DigdagTimedMethodInterceptor.invoke(DigdagTimedMethodInterceptor.java:31)
        at io.digdag.core.agent.OperatorManager.runWithWorkspace(OperatorManager.java:308)
        at io.digdag.server.metrics.DigdagTimedMethodInterceptor.invokeMain(DigdagTimedMethodInterceptor.java:58)
        at io.digdag.server.metrics.DigdagTimedMethodInterceptor.invoke(DigdagTimedMethodInterceptor.java:31)
        at io.digdag.core.agent.OperatorManager.lambda$runWithHeartbeat$2(OperatorManager.java:152)
        at io.digdag.core.agent.ExtractArchiveWorkspaceManager.withExtractedArchive(ExtractArchiveWorkspaceManager.java:75)
        at io.digdag.core.agent.OperatorManager.runWithHeartbeat(OperatorManager.java:150)
        at io.digdag.server.metrics.DigdagTimedMethodInterceptor.invokeMain(DigdagTimedMethodInterceptor.java:58)
        at io.digdag.server.metrics.DigdagTimedMethodInterceptor.invoke(DigdagTimedMethodInterceptor.java:31)
        at io.digdag.core.agent.OperatorManager.run(OperatorManager.java:133)
        at io.digdag.server.metrics.DigdagTimedMethodInterceptor.invokeMain(DigdagTimedMethodInterceptor.java:58)
        at io.digdag.server.metrics.DigdagTimedMethodInterceptor.invoke(DigdagTimedMethodInterceptor.java:31)
        at io.digdag.core.agent.MultiThreadAgent.lambda$null$0(MultiThreadAgent.java:132)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)
2024-09-10 08:21:56.680 +0000 [INFO] (0066@[0:p_5dc01c89-8aac-44e3-b2c1-2871ff90392e_1]+w_1^failure-alert) io.digdag.core.agent.OperatorManager: type: notify

I suppose it is correct to break on this line when the url parameter is specified.

embulk-output-jdbc/embulk-output-sqlserver/src/main/java/org/embulk/output/SQLServerOutputPlugin.java

Line 204 in 99d8603

url = sqlServerTask.getUrl().get();

I think it is correct to implement the same as the input plugin.
https://github.com/embulk/embulk-input-jdbc/blob/3f7fb40cfb9814b0214325141cf3efffccd3c084/embulk-input-sqlserver/src/main/java/org/embulk/input/sqlserver/SQLServerInputPlugin.java#L217
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kei-hito