Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

request: add loop benchmark #415

Open
amlan-sw opened this issue Jul 24, 2023 · 2 comments
Open

request: add loop benchmark #415

amlan-sw opened this issue Jul 24, 2023 · 2 comments

Comments

@amlan-sw
Copy link

amlan-sw commented Jul 24, 2023

sometime we don't need complicated algorithm to see performance of programming
language, we just need to see how these language loop's implemetation, all benchmark
game like binarytree, fasta, etc just only test loop implementation of language and sometime
benchmark implementation syscall like 'print' to console through loop

example just loop for 1000_000_000 (billion) in linux bash for various language,
i'm using digitalocean cloud with centos7

perl 5.16 -> 5 secs just loop

# time perl -e 'my $a = 1; for $i (0 .. 1000000000){$a+=4}; print $a'
4000000005
real    0m5.020s
user    0m5.006s
sys     0m0.009s
---  php 5.4.16 -> 3.5 secs just loop

# time php -r '$a = 1; for($i=0;$i<=1000000000;$i++){ $a+=4; } echo $a;'
4000000005
real    0m3.536s
user    0m3.502s
sys     0m0.032s

--- php 8.2.8 -> 1.2 secs just loop

# time php82 -r '$a = 1; for($i=0;$i<=1000000000;$i++){ $a+=4; } echo $a;'
4000000005
real    0m1.181s
user    0m1.152s
sys     0m0.029s
python

--- src

# cat loop.py
a = 1
for i in range(0,1000000001):
    a += 4

print(a)

--- python2 -> error just loop

# time python loop.py
Traceback (most recent call last):
  File "loop.py", line 2, in <module>
    for i in range(0,1000000000):
MemoryError

--- python3 -> 30 secs just loop

# time python3 loop.py
4000000005
real    0m30.887s
user    0m30.857s
sys     0m0.015s
C gcc 11.2.1

--- src

# cat loop.c

#include <stdio.h>
void main(){
        long i,a=1;
        for(i=0;i<=1000000000;i++){
                a += 4 ;
        }
    printf("%lli",a);
}

--- default optimation -> 0.5 secs just loop

# gcc -o loop loop.c
# time loop
4000000005
real    0m0.526s
user    0m0.523s
sys     0m0.003s


--- O3 optimation -> 0.002 secs just loop
    
# gcc -O3 -o loop loop.c
# time loop
4000000005
real    0m0.003s
user    0m0.001s
sys     0m0.002s

note:
we need simple operation on loop like a+=4 so the optimizer of language 'dont cheat' the output, because empty operation might make some language optimizer skip loop operation

@igouy
Copy link

igouy commented Oct 26, 2023

Here's a too simple loop — https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/toosimple.html

Notice that swapping the order of two statements can have a dramatic impact.

Notice that using global vars vs local vars can have a dramatic impact.

Notice that the particular language implementation can have a dramatic impact.

@JZerf
Copy link
Contributor

JZerf commented Oct 29, 2023

note:
we need simple operation on loop like a+=4 so the optimizer of language 'dont cheat' the output, because empty operation might make some language optimizer skip loop operation

You should use a more complex operation in the loop that can't be optimized out. A good, optimizing compiler/interpreter will have little issue with optimizing out a loop that just performs increments using a constant value. As demonstrated by your timings, GCC had no issue optimizing out the loop when optimization was enabled and completed the program in just 0.003s which would be impossible on all current microprocessors if it had actually ran that loop 1,000,000,001 times.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants