How can we get data from db more than 10,000

Studio
1

I need to get more than 10,000 from DB. the bot is using app.yellow.ai. please suggest
the usecase is I will get the data from db and generate the data to csv file. the csv file will be uploaded to client AWS S3 bucket

2

You cannot get more than 10K from DB

But you can create pagination over DB using a timestamp with sorting. you can read more in ElasticSearch docs about how to write queries with sorting, etc…

1 Like
3

@Harish_Mukkara can I know how to have pagination on elastic search?

4

also got this error
[illegal_argument_exception] Result window is too large, from + size must be less than or equal to: [10000] but was [10001]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.