scans/ rescans
Posted: 05 Oct 2014, 01:31
a few remarks on my personal experience lately. i use binhex-madsonic docker on an unraid server, before i used botez-madsonic on the same server... even before that i used a subsonic docker on the same machine.
i have a largish music only collection of about 12tb / 100k albums. under subsonic a scan used to take about 3hrs. since i changed to madsonic i am looking at numbers in the area 30+hrs when i used the botez docker and around 15hrs with the binhex docker. what is quite a drastic change to the worse.
i have to say the improvement in the gui responsiveness are quite impressive with the last 5.1 final releases. so my thanx for that and the many other little improvements. generally my big thanx for madsonic in general - it makes subsonic so much more useful!!!!
anyway, doing some research and thinking about the prob - i have some questions / suggestions. since my programming skills and knowledge about the workings of database engines are rudimentary at best, plz correct me where i am wrong or consider it for future releases.
- first i noticed a huge discrepancy between reads and writes during a scan (there are about 7 writes for every read) i assume this is due to the extended functions of madsonic vs. subsonic!? since much of it should still be based on the same information in tags etc. - is there a big room to optimize the db tables?
- ms and ss are build with hypersql which is based on java, can that be a limiting factor in performance? java to my knowledge is not really known for speed and often enough causes when used by multiple programs at least under linux crashes (which i didn't have yet) and minor probs.
- i read during my searches for solutions about increasing the memory limit, i did that to about 1500mb - no noticeable real improvement there. i read also somewhere on the ss forums a post of somebody who undertook for himself a change/ rewrite from hypersql to postgresql claiming major performance increase.
now to the actual request/ question - would a db change from hypersql to postgresql be likely to accomplish that? if so, why not switch for better performance? how complicated would it be? is it something that could be considered for future releases / has anybody attempted that for madsonic and has some information how performance compares?
i think also something must be fundamentally wrong when a simple rescan (just add the new stuff) takes as long as a full scan of your media (they do- i tried it). a simple rescan should take a substantially shorter time. a good example for that is kplaylist (which i used for many years before switching to ss and recently ms a couple years back due to the stagnation in the kplaylist development). there a full scan used to take about 2hrs, a rescan as little as 5min!!!
seems like a much better deployed db functionality to me as the layman.
looking forward to answers, explanations, ideas or commitments of ppl with the ability to do it to find out
cheers, L
i have a largish music only collection of about 12tb / 100k albums. under subsonic a scan used to take about 3hrs. since i changed to madsonic i am looking at numbers in the area 30+hrs when i used the botez docker and around 15hrs with the binhex docker. what is quite a drastic change to the worse.
i have to say the improvement in the gui responsiveness are quite impressive with the last 5.1 final releases. so my thanx for that and the many other little improvements. generally my big thanx for madsonic in general - it makes subsonic so much more useful!!!!
anyway, doing some research and thinking about the prob - i have some questions / suggestions. since my programming skills and knowledge about the workings of database engines are rudimentary at best, plz correct me where i am wrong or consider it for future releases.
- first i noticed a huge discrepancy between reads and writes during a scan (there are about 7 writes for every read) i assume this is due to the extended functions of madsonic vs. subsonic!? since much of it should still be based on the same information in tags etc. - is there a big room to optimize the db tables?
- ms and ss are build with hypersql which is based on java, can that be a limiting factor in performance? java to my knowledge is not really known for speed and often enough causes when used by multiple programs at least under linux crashes (which i didn't have yet) and minor probs.
- i read during my searches for solutions about increasing the memory limit, i did that to about 1500mb - no noticeable real improvement there. i read also somewhere on the ss forums a post of somebody who undertook for himself a change/ rewrite from hypersql to postgresql claiming major performance increase.
now to the actual request/ question - would a db change from hypersql to postgresql be likely to accomplish that? if so, why not switch for better performance? how complicated would it be? is it something that could be considered for future releases / has anybody attempted that for madsonic and has some information how performance compares?
i think also something must be fundamentally wrong when a simple rescan (just add the new stuff) takes as long as a full scan of your media (they do- i tried it). a simple rescan should take a substantially shorter time. a good example for that is kplaylist (which i used for many years before switching to ss and recently ms a couple years back due to the stagnation in the kplaylist development). there a full scan used to take about 2hrs, a rescan as little as 5min!!!
seems like a much better deployed db functionality to me as the layman.
looking forward to answers, explanations, ideas or commitments of ppl with the ability to do it to find out
cheers, L