Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to create CSV file with 20000 rows #1

Open
A3a3e1 opened this issue Nov 9, 2021 · 0 comments
Open

Failed to create CSV file with 20000 rows #1

A3a3e1 opened this issue Nov 9, 2021 · 0 comments

Comments

@A3a3e1
Copy link
Contributor

A3a3e1 commented Nov 9, 2021

Below is the issue elasticio/csv-component#47
Need to check if it is still a bug


I've build a simple integration flow:

image

then using simple load-test tool loaded 2000 request to it:

#!/bin/bash
loadtest -c 5 -n 20000 "https://in.elastic.io/hook/5de5199b20ea507fdaf1a542?message=Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum"

It run all well:

[Mon Dec 02 2019 15:34:23 GMT+0100 (Central European Standard Time)] INFO Max requests:        20000
[Mon Dec 02 2019 15:34:23 GMT+0100 (Central European Standard Time)] INFO Concurrency level:   5
[Mon Dec 02 2019 15:34:23 GMT+0100 (Central European Standard Time)] INFO Agent:               none
[Mon Dec 02 2019 15:34:23 GMT+0100 (Central European Standard Time)] INFO
[Mon Dec 02 2019 15:34:23 GMT+0100 (Central European Standard Time)] INFO Completed requests:  20000
[Mon Dec 02 2019 15:34:23 GMT+0100 (Central European Standard Time)] INFO Total errors:        0
[Mon Dec 02 2019 15:34:23 GMT+0100 (Central European Standard Time)] INFO Total time:          837.860013091 s
[Mon Dec 02 2019 15:34:23 GMT+0100 (Central European Standard Time)] INFO Requests per second: 24
[Mon Dec 02 2019 15:34:23 GMT+0100 (Central European Standard Time)] INFO Mean latency:        209.4 ms
[Mon Dec 02 2019 15:34:23 GMT+0100 (Central European Standard Time)] INFO
[Mon Dec 02 2019 15:34:23 GMT+0100 (Central European Standard Time)] INFO Percentage of the requests served within a certain time
[Mon Dec 02 2019 15:34:23 GMT+0100 (Central European Standard Time)] INFO   50%      201 ms
[Mon Dec 02 2019 15:34:23 GMT+0100 (Central European Standard Time)] INFO   90%      213 ms
[Mon Dec 02 2019 15:34:23 GMT+0100 (Central European Standard Time)] INFO   95%      235 ms
[Mon Dec 02 2019 15:34:23 GMT+0100 (Central European Standard Time)] INFO   99%      400 ms
[Mon Dec 02 2019 15:34:23 GMT+0100 (Central European Standard Time)] INFO  100%      723 ms (longest request)

at the end I expected a file to be uploaded to SFTP to have 20.000 rows in it:

[2019-12-02 14:34:33.287]: The resulting CSV file contains 20000 rows
[2019-12-02 14:34:33.287]: Closing the stream due to inactivity
[2019-12-02 14:34:33.290]: Emitting message {"id":"f705ff7c-b18a-4e91-83ac-5a879745d92b","attachments":{"f705ff7c-b18a-4e91-83ac-5a879745d92b.csv":{"content-type":"text/csv","url":"http://steward-service.platform.svc.cluster.local:8200/files/0375788e-5fcd-4931-8208-811c28d1ac3d"}},"body":{"rowCount":20000},"headers":{},"metadata":{}}

unfortunately resulting file was only 1 MB large and had 1795 rows in it, not 20000

Attaching result and log of the execution as ZIP file.

Archive.zip

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant