Checkpointing: My understanding says checkpoints comes into play while applying rows to the DB. Not sure if file reader makes use of any checkpoint attribute. However, we are not specifying any checkpoint.
1 million files in a single directory: Well no, we can distribute across multiple directories on the same volume.
No. of Rows per file: wide range of values depending upon the size of the file. 500KB is just a average figure. 100 rows / 500KB
Are you using the Load operator?: Tried using Load as well as Stream operator.
How long are you *anticipating* the acquisition phase taking?: if we extrapolate the results, whole processing may take around 40-50 hrs, which doesn't seem appropriate.
Frequency: monthly
We tried using upto 10 readers, however throughput was still the same.
↧