you'll know better than me from the code side but would it be worth trying (if possible) combining your latest code tweaks together with the unbuffered just to see if that speeds things up even more or not? Might help to clarify exactly which things are causing/fixing the issue? But if not then this latest version is looking good anyway, have you had a chance to see how it performs under Windows/Linux as yet? So I think we're close with your latest code tweaks. Looks like the latest is pretty close to the first (unbuffered) one although seems slightly less consistent perhaps? Just run some tests with original v8.8, 9.6 first beta and latest 9.6 beta you posted yesterday. This could be explained if on macOS (which is built upon BSD), the Posix API itself was suboptimal and built on top of more performant lower-level access routines, which "Finder" is using. it does not on Linux, Windows) but seems to be a general issue with the Posix API providing C-level streams on macOS. This implies that the perf issue in general is not only related to the application layer buffering (I still don't understand why this pessimization exists, while e.g. Other means to copy files like "cp" or "native" copy commands (which technically are implemented similarly like "cp") such as "fcopyfile" are equally slow. Unbuffered file copy is 30-40% faster.īut: even this faster version is still *twice* as slow as file copy via "Finder" and also twice as slow as the Linux file copy test case on the same network share same results for local file copy. macOS: Due to technical limitations I wasn't able to test "variable/small data blocks" yet. Linux: Unbuffered write is 22% slower for variable/small data blocks, when the target volume is a network share, it's 3400% slower. Windows: Unbuffered write is 80% slower for variable/small data blocks (e.g. Maybe the use of SSDs has changed that and uncovered a new bottleneck, that would not have mattered for lower disk access speeds.Īfter testing on Linux and Windows, it seems the SSD-explanation does not hold:ġ. Considering that hard-drives operate(d) at a speed that was magnitudes slower than memory, it shouldn't matter whether data would be copied around for buffering purposes.
In the case of file copying there is not really much benefit since the data blocks that are copied are reasonably large, but what's really surprising is that 3 can actually be a perf pessimization. An application-level buffer 3 is standard procedure for file-copying purposes and optimizes potential overhead when calling into the OS too often and for too little bits of data. This would be slow if there weren't the other buffers 1 and 2. The last beta you tested basically got rid of the 3rd buffer, and did what is called "unbuffered I/O". buffering at the hardware level, managed by the hard disk Perf bugs are one of the most difficult ones to fix, since usually nobody reports them (FFS 8.9 is 9 month old!) and even when confirmed, you're lucky when there is only a single cause, like we have here.ĭuring file copy, three types of buffers are involved:ġ. The credit mostly goes to you, for reporting this issue in first place and doing the testing.
#FREEFILESYNC RUNNING VEY SLOW ON MAC CRACKED#
Well I think you've cracked it! :) DerekPap, , 10:07 Is it safe for me to continue using the 9.6 beta you posted or should I wait until an official 9.6 release? without SSD to SSD) it was significant to be noticed until now?Īnyway thanks for finding the solution, good work! I guess the question being now is the "additional memory copies that were added" you mentioned were presumably added to improve something (?) and hence by removing the file buffering will that be degrading for other scenarios? Or was it possible that it did degrade other scenarios but just not noticed as at slower speeds in general (i.e. V9.6 - 43 files / 6.16GB - 449GB / 14 secondsĪs you can see I'm now getting around 448MB/sec vs 352MB/sec (with v8.9 it was about 226MB/sec) so really good. Well I think you've cracked it! :) Doing some tests (see below) comparing v8.8 with v9.6 beta not only has the problem gone away but it's now significantly faster (rather than slower) than v8.8 which I guess is due to the other improvements/updates made between 8.8 -> 9.6?
0 kommentar(er)
