Cygwin multithreading performance
Kacper Michajlow
kasper93@gmail.com
Sat Dec 5 13:59:00 GMT 2015
2015-12-05 14:07 GMT+01:00 Kacper Michajlow <kasper93@gmail.com>:
> 2015-12-05 11:51 GMT+01:00 Mark Geisert <mark@maxrnd.com>:
>> Mark Geisert wrote:
>>>
>>> Corinna Vinschen wrote:
>>>>
>>>> On Nov 23 16:54, Mark Geisert wrote:
>>>>>
>>>>> John Hein wrote:
>>>>>>
>>>>>> Mark Geisert wrote at 23:45 -0800 on Nov 22, 2015:
>>>>>> > Corinna Vinschen wrote:
>>>>>> > > On Nov 21 01:21, Mark Geisert wrote:
>>>>>> > [...] so I wonder if there's
>>>>>> > >> some unintentional serialization going on somewhere, but I
>>>>>> don't know yet
>>>>>> > >> how I could verify that theory.
>>>>>> > >
>>>>>> > > If I'm allowed to make an educated guess, the big serializer
>>>>>> in Cygwin
>>>>>> > > are probably the calls to malloc, calloc, realloc, free. We
>>>>>> desperately
>>>>>> > > need a new malloc implementation better suited to
>>>>>> multi-threading.
>>>
>>> [...]
>>>>>>
>>>>>>
>>>>>> Someone recently mentioned on this list they were working on porting
>>>>>> jemalloc. That would be a good choice.
>>>>>
>>>>>
>>>>> Indeed; thanks for the reminder. Somehow I hadn't followed that thread.
>>>>
>>>>
>>>> Indeed^2. Did you look into the locking any further to see if there's
>>>> more than one culprit? I guess we've a rather long way to a "lock-less
>>>> kernel"...
>>
>> [...]
>>>
>>> But that is just groundwork to identifying which locks are suffering the
>>> most contention. To identify them at source level I think I'll also
>>> need to record the caller's RIP when they are being locked.
>>
>>
>> In the OP's very good testcase the most heavily contended locks, by far, are
>> those internal to git's builtin/pack-objects.c. I plan to show actual stats
>> after some more cleanup, but I did notice something in that git source file
>> that might explain the difference between Cygwin and MinGW when running this
>> testcase...
>>
>> #ifndef NO_PTHREADS
>>
>> static pthread_mutex_t read_mutex;
>> #define read_lock() pthread_mutex_lock(&read_mutex)
>> #define read_unlock() pthread_mutex_unlock(&read_mutex)
>>
>> static pthread_mutex_t cache_mutex;
>> #define cache_lock() pthread_mutex_lock(&cache_mutex)
>> #define cache_unlock() pthread_mutex_unlock(&cache_mutex)
>>
>> static pthread_mutex_t progress_mutex;
>> #define progress_lock() pthread_mutex_lock(&progress_mutex)
>> #define progress_unlock() pthread_mutex_unlock(&progress_mutex)
>>
>> #else
>>
>> #define read_lock() (void)0
>> #define read_unlock() (void)0
>> #define cache_lock() (void)0
>> #define cache_unlock() (void)0
>> #define progress_lock() (void)0
>> #define progress_unlock() (void)0
>>
>> #endif
>>
>> Is it possible the MinGW version of git is compiled with NO_PTHREADS
>> #defined? If so, it would mean there's no locking being done at all and
>> would explain the faster execution and near 100% CPU utilization when
>> running under MinGW.
>
> Nah, there is no threading enabled when there is no pthreads. How
> would that work? :D See thread-utils.h
>
> #ifndef NO_PTHREADS
> #include <pthread.h>
>
> extern int online_cpus(void);
> extern int init_recursive_mutex(pthread_mutex_t*);
>
> #else
>
> #define online_cpus() 1
>
> #endif
>
>
> Looks like there is indeed a bug in git code when passing "--threads"
> explicitly to "git pack-objects", because they show warning about
> threads being unsupported, but doesn't overwrite delta_search_threads
> value. I will go to git's ML about it. This is completely not related
> to our issue.
Obviously I was wrong. There is
#define ll_find_deltas(l, s, w, d, p) find_deltas(l, &s, w, d, p)
So 'delta_search_threads' value is never used. Still not related to
cygwin issue tho ;)
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
More information about the Cygwin
mailing list