How can we get data from db more than 10,000

I need to get more than 10,000 from DB. the bot is using please suggest
the usecase is I will get the data from db and generate the data to csv file. the csv file will be uploaded to client AWS S3 bucket

You cannot get more than 10K from DB

But you can create pagination over DB using a timestamp with sorting. you can read more in ElasticSearch docs about how to write queries with sorting, etc…

1 Like

@Harish_Mukkara can I know how to have pagination on elastic search?

also got this error
[illegal_argument_exception] Result window is too large, from + size must be less than or equal to: [10000] but was [10001]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.