Members of huge ASCII DataSet not shown
Author: Ben_Zurich
Creation Date: 6/15/2015 9:31 AM
profile picture

Ben_Zurich

#1
We recently have boosted one of our dataset folders containing .csv files from about 19'000 to over 28'000 files. Since then, the folder appears in the WL6 dataset list but cannot be expanded and the files cannot be accessed.

Is this an intended limit ? Per folder? Overall maximum files ? Overall maximum size of all files?

(I have also made a ticket)
profile picture

Eugene

#2
Duplicating and expanding my answer from the closed ticket:

There's no intended limit in the ASCII provider. Sounds like you have maxed out some of your computer's resources (e.g. available RAM). I'm not sure if TreeView control has any limits but instead of pushing RAM / GDI handle limitations, try to make it easier for the computer:

1. Consider using an appropriate tool for big data instead of ASCII files - such as the Database provider.
2. Otherwise break up the data folder in manageable chunks e.g. 3000 files.