_check_sigspec is slow

[uriva]

Hi, while optimizing some code I noticed an unreasonable amount of the time is spent in _check_sigspec. I suspect this is the heavy usage of inspect methods in curry code.

I think this is not justified as one can run inspect stuff once, and store for later.

Did anyone encounter this?

[uriva]

2224/1	0.01107	0.01107	3.424	3.424	html_parser.py:200(_parse)
3592/1	0.004297	0.004297	3.423	3.423	functional_generic.py:162(pipe)
3981/2	0.003494	0.001747	3.423	1.711	functional_generic.py:125(ternary_inner)
110576	0.1263	1.142e-06	2.802	2.534e-05	functoolz.py:301(call)
31393	0.07909	2.519e-06	2.317	7.382e-05	functoolz.py:309(_should_curry)
116488/65010	0.1243	1.911e-06	1.546	2.378e-05	functoolz.py:829(_check_sigspec)
12498/10521	0.00642	6.102e-07	1.422	0.0001351	functional.py:99(star_and_run)
9315/7713	0.02187	2.836e-06	1.266	0.0001642	functional_generic.py:166(_curry_helper)
2934/1332	0.007844	5.889e-06	1.209	0.0009078	html_parser.py:183(_shift_reduce)
297127/177935	0.05967	3.353e-07	1.188	6.675e-06	~:0()
71689/42691	0.02959	6.931e-07	1.186	2.778e-05	itertoolz.py:370(first)
1368	0.007718	5.642e-06	1.175	0.000859	html_parser.py:166(_reduce)
79441	0.04964	6.249e-07	1.173	1.477e-05	inspect.py:3091(signature)
3826/3219	0.00591	1.836e-06	1.149	0.0003569	functional_generic.py:209(indirection)
79441	0.05162	6.497e-07	1.124	1.415e-05	inspect.py:2839(from_callable)
79443/79442	0.4248	5.348e-06	1.072	1.35e-05	inspect.py:2206(_signature_from_callable)
61964/30857	0.03227	1.046e-06	1.043	3.38e-05	functional.py:306(apply_inner)
57132/31393	0.06314	2.011e-06	0.9531	3.036e-05	functoolz.py:917(is_partial_args)
7919	0.005044	6.369e-07	0.8553	0.000108	html_parser.py:157(_apply)
51478/25739	0.09437	3.666e-06	0.6487	2.52e-05	functoolz.py:891(has_varargs)
65483/65077	0.03838	5.898e-07	0.5533	8.502e-06	~:0()
25739	0.03523	1.369e-06	0.4799	1.864e-05	_signatures.py:643(signature_or_spec)
25739	0.04606	1.79e-06	0.4523	1.757e-05	_signatures.py:756(_is_partial_args)
51478	0.01648	3.201e-07	0.3955	7.683e-06	_signatures.py:761()
25739	0.03391	1.318e-06	0.379	1.473e-05	_signatures.py:743(check_partial)
37047	0.2379	6.423e-06	0.3661	9.882e-06	inspect.py:2889(_bind)
31393	0.03032	9.659e-07	0.3496	1.114e-05	functoolz.py:335(bind)
31393	0.02092	6.664e-07	0.3435	1.094e-05	inspect.py:3027(bind_partial)
31393	0.1279	4.075e-06	0.3193	1.017e-05	functoolz.py:194(init)
4568/4190	0.00218	5.203e-07	0.2955	7.051e-05	functoolz.py:779(call)
25739	0.03551	1.379e-06	0.2534	9.847e-06	_signatures.py:788(_has_varargs)
25739	0.01406	5.463e-07	0.2111	8.2e-06	_signatures.py:792()
25739	0.01684	6.543e-07	0.197	7.654e-06	_signatures.py:783(check_varargs)
79441	0.09083	1.143e-06	0.1863	2.345e-06	inspect.py:493(unwrap)

[uriva]

to whom it may concern, gamla is a drop in replacement for most of what toolz does, and does it considerably faster by avoiding issues such as this.

[groutr]

@uriva can you share the code that you were profiling? It would make it easier to understand what is happening.

[uriva]

It's been awhile, but iirc it was just using a curried function in a loop, or building a composition using compose in a loop.