Performance slow

Posted over 4 years ago by Vin Singh

Post a topic

Un Solved

Vin Singh

Hello all,

I have been tasked over the last several months to do a major search and to find the right program to do it. I have tried a handful of them out there free/open or as trials but all of them have had issues with speed or not able to perform searches as it has been requested. I just recently tried Autopsy, really nice but no search operators and it seems to have overworked the external drive and broke it (lol).

Due to all these issues I transferred all the data to one of my physical Lab server which is an old Dell Poweredge 710 server. This is much more capable than then desktop and the laptop I was using before.
System Spec:
-TWO (2) x Intel Xeon CPU E5620 @ 2.40GHz (4 cores each cpu/8 total/16 total logical)
-144GB DDR EC RAM
-2 x hard drives setup as RAID 0 specifically for this data to double read/write

Copernic im finding very slow on the index, pretty much the same as on a mid range laptop. Initially I loaded the entire system drives (C:) but since it was taking so long I only included the C:\users folder from each system (about 1.5mil files) and loaded 150GB across 20 PST files.

Its been almost a week of Copernic running and it has been indexing the PST files so far and it has only made it to 140k index. I went through every performance setting and also it is running in unrestricted mode. I see that Copernic will use at most 10-15% of the CPU and memory is 15GB used of 144GB.

Is this all that the software is capable of or is there something I am doing wrong? I did buy a license for use on my bosses computer but I am running this specific install as a Trial before committing to it. Is the Trial the cause? I was sure I read it was fully functional.

Any help is appreciated.

0 Votes

8 Comments

Sean Toomey posted about 3 years ago

Ver 8. Build 14438

I have used this app for years. Never this slow. Reindexing performance is very slow. Using outlook from Microsoft office 365 local install. Files on local c drive

0 Votes

Andreas Korntheuer posted almost 4 years ago

hi team,

after one week indexing my server only indexed aprox 500.000 files.

I disabled ocr scanning after 2 days. But the scan is not realy fast.

Also the setting of ExtractionThreadPoolSize to 4 und CrawlingThreadPoolSize to 2.

After disabling jpg / picture option to scan the content it seems to be faster.

What information is scaned if copernic server scan the "content" of jpg? Does it index the exif data?

Same with some other file extensions from category pictures and movies.

What does copernic server index if scanning content is activated?

- mov, mp4, mp3, mpeg ...

- ogg

- png, psb, psd

regards

Andreas Korntheuer

0 Votes

Mirco Persechino posted about 4 years ago

Hi JD,

As mentionned, it is not recommended to exceed 4 in the ExtractionThreadPoolSize.

In the OCR options tab, did you checked only languages you're actively using or everything is checked? Because this factor can drastically affect performances if you set languages that you don't necessarily have in your documents, because CDS will always try to scan those files and try every language available to see if it fits.

So only select languages that you know you're having in your files.

I'll report this performance issue to our development team. You can also let us know what kind of CPU and RAM memory you're having on your computer to help us in diag :-)

Thanks!

1 Votes

LUST & Partners Jacky d'Hoest posted about 4 years ago

I also complained about speed issues and I applied this message from Max L. :

"The indexing options are configured for an average desktop computer. By default, CDS is configured to run only one "thread" to read the sources and two "threads" to extract the content. These settings can be adjusted in the config.xml file from your Copernic Desktop Search install folder but this should not be considered as a common operation to modify those settings (Options "CrawlingThreadPoolSize" and "ExtractionThreadPoolSize" in the configuration file config.xml).

On a very powerful machine, you can play a bit with "ExtractionThreadPoolSize", but it can end up slowing down if the number of threads is too high as the extractors can't execute as many documents. We don't recommend to exceed 4 or 6 threads. For "CrawlingThreadPoolSize", no more than 2."

To my surprise on start up CDS changes this settings from the recommended 2 - 6 to 2 - 4 and resaves the config.xml file. What is the reason for that ? Is there another parameter that must be changed to imply this correctly ?

Can we stil change something else / another parameter to fasten up CDS ?

In anyway using the OCR-plug-inn on my PC slows down CDS by a factor of 20. A full update which takes 4 days suddenly takes up to one month !

Y.s.

0 Votes

Dean C posted about 4 years ago

Suddenly, no displays after "loading extensions" msg. exits. Have been routinely been using for years without ANY problems. Now this!

Any/all comments and suggestions greatly appreciated!!

0 Votes

Brian Walters posted about 4 years ago

I have been using the paid version since July 2015 and find it very slow to search and often shows the number of matches in the file type icons but will not display any of the matched files in the default "all results" icon. The index is updated unrestricted for 8 to 12 hours every night and pause it when I start working. I usually exit the program due to the number of resources it uses and only open it when needed to find something that includes OCR and MS Outlook emails. Otherwise, I use a free program "Everything " by voidtools.com which indexes using file name only with instant results. The first time you start it takes less than one minute to index all local drives with around 2 million files. If Desktop Search was so slow to show results, I would use it all the time but 20 to 30 seconds waiting for results and again every time you change the file type icon usually because the results are blank in the main results screen.

0 Votes

Mirco Persechino posted about 4 years ago

Hi!

The Trial version is like a Retail except for the duration of the license. There is no speed limitation due to a trial version.

If you have the OCR text recognition option enabled, this option slows down indexing considerably since the pictures content in PDF and Word documents are scanned in order to extract any recognized text. This could affect performance during initial index.

The indexing options are configured for an average desktop computer. By default, CDS is configured to run only one "thread" to read the sources and two "threads" to extract the content. These settings can be adjusted in the config.xml file from your Copernic Desktop Search install folder but this should not be considered as a common operation to modify those settings (Options "CrawlingThreadPoolSize" and "ExtractionThreadPoolSize" in the configuration file config.xml).
On a very powerful machine, you can play a bit with "ExtractionThreadPoolSize", but it can end up slowing down if the number of threads is too high as the extractors can't execute as many documents. We don't recommend to exceed 4 or 6 threads. For "CrawlingThreadPoolSize", no more than 2.

Regarding PSTs indexed outside of Microsoft Outlook : If you're indexing large PST files outside of your Microsoft Outlook profile, it can take long to complete indexing. Each PST file must be opened multiple times by Copernic Desktop Search to read the content and extract each item. (A 10GB PST would take much longer to index than 2 separate files of any kind of 5Gb each for example).

Hope it helps.

1 Votes

wacocrimatty posted about 4 years ago

I am having the same issue. I find the indexing occurs in spurts, then starts again. I have used this program for years, and have found it to be satisfactory. However, this speed issue is quite frustrating.

BTW--THIS SAYS THE ISSUE IS "SOLVED" BUT I DON'T SEE THE SOLUTION.

0 Votes