LoongArch to Experimental tier (CI environment)

[shipujin]

I would like to discuss here how to put LoongArch machine access nodejs/build CI environment, test machine for LoongArch nodejs.

Best regards

[shipujin]

@sxa #3511 (comment)
Sorry, I've been testing nodejs(main v18. x v20.x), reply late

The question in my mind is what are we expecting to gain from adding it to the CI?

LoongArch platform can follow up the latest status of node, solve problems in the process, and promote the development of loong64-node.

1. What is the outlook for getting the tests passing cleanly? If it was added into the regular CI then we'd want it to run without errors otherwise it will cause problems interpreting the test results potentially resulting in delays to validating PRs.

OS: LoongArch Fedora remix

latest v18/v20/main

wget -c https://nodejs.org/download/release/latest-v18.x/node-v18.18.2.tar.gz
wget -c https://nodejs.org/download/release/latest-v20.x/node-v20.9.0.tar.gz
git clone https://github.com/nodejs/node.git

may need to apply this patch

nodejs/node#50209
nodejs/node#50363
#v20.x: may need to apply this patch
nodejs/node#50210

build

./configure --openssl-no-asm
time make -j4

check

export OPENSSL_CONF="./deps/openssl/openssl/apps/openssl.cnf"
time make test-ci-js 2>&1 | tee ../E.log-nodejs-ci

latest-v18/latest-v20/main

The latest test-ci-js version of v18/v20/main failed 3 tests

test/parallel/test-crypto-sign-verify.js
test/parallel/test-vm-global-setter.js
test/parallel/test-vm-module-synthetic.js

2. What is the performance of the systems compared to the other architectures (since that affects the execution time of CI jobs)? I suspect that it us unlikely that we could integrate it into the regular CI without having a significant negative impact on the run times of the PR testing.

latest-v18

(1) time make -j4

...
real    87m53.900s
user    215m25.687s
sys     6m22.889s

(2) time make test-ci-js 2>&1 | tee ../E.log-nodejs-ci

...
Failed tests:
out/Release/node /home/shipujin/node-v18.18.2/test/parallel/test-crypto-sign-verify.js
out/Release/node /home/shipujin/node-v18.18.2/test/parallel/test-vm-global-setter.js
out/Release/node --experimental-vm-modules /home/shipujin/node-v18.18.2/test/parallel/test-vm-module-synthetic.js
make: *** [Makefile:539：test-ci-js] Error 1

real    25m25.999s
user    37m11.585s
sys     1m44.890s

latest-v20

(1) time make -j4

...
real    95m7.574s
user    247m31.239s
sys     6m45.546s

(2) time make test-ci-js 2>&1 | tee ../E.log-nodejs-ci

...
Failed tests:
out/Release/node /home/shipujin/LTS/node-v20.9.0/test/parallel/test-crypto-sign-verify.js
out/Release/node /home/shipujin/LTS/node-v20.9.0/test/parallel/test-vm-global-setter.js
out/Release/node --experimental-vm-modules /home/shipujin/LTS/node-v20.9.0/test/parallel/test-vm-module-synthetic.js
make: *** [Makefile:535: test-ci-js] Error 1
                                                                    
real    21m28.480s
user    34m21.144s
sys     3m20.023s

main

(1) time make -j4

...

real    93m59.754s
user    354m6.669s
sys     9m37.371s

(2) time make test-ci-js 2>&1 | tee ../E.log-nodejs-ci

...
Failed tests:
out/Release/node /home/shipujin/node/test/parallel/test-crypto-sign-verify.js
out/Release/node /home/shipujin/node/test/parallel/test-vm-global-setter.js
out/Release/node --experimental-vm-modules /home/shipujin/node/test/parallel/test-vm-module-synthetic.js
make: *** [Makefile:535：test-ci-js] Error 1
                                                                    
real    27m55.096s
user    37m24.999s
sys     3m21.715s

Having said that, as Michael says, I'm not against having machines available for interested parties to progress the status of the port and work towards having it in place, and I can't think of any objection, given the assurances above, that it could be moved to being listed as an experimental platform.

There are two questions about the way LoongArch machine is provided, hope you can help me, thanks~
(1) How are LoongArch machines provided to nodejs?
I can provide a LoongArch machine, the machine runs in my company's machine room. Later I can provide an ssh access to the LoongArch machine ip, is this allowed?
(2) How to connect LoongArch machine to nodejs/build project for CI test?

Best regards

@nodejs/build @mhdawson @richardlau @sxa @targos

[targos]

(2) How to connect LoongArch machine to nodejs/build project for CI test?

The machine must be able to run the Jenkins agent (Java 11 or 17)

[sxa]

(2) How to connect LoongArch machine to nodejs/build project for CI test?

The machine must be able to run the Jenkins agent (Java 11 or 17)

21'should also be acceptable now - I can't remember if the loongarch support was backported to JDK17

[shipujin]

(2) How to connect LoongArch machine to nodejs/build project for CI test?

The machine must be able to run the Jenkins agent (Java 11 or 17)

Yes, the running system is java17.

[shipujin@bogon node]$ 
[shipujin@bogon node]$ java --version
openjdk 17.0.5 2022-10-18
OpenJDK Runtime Environment (Red_Hat-17.0.6.0.1-2.fc38) (build 17.0.5+1)
OpenJDK 64-Bit Zero VM (Red_Hat-17.0.6.0.1-2.fc38) (build 17.0.5+1, interpreted mode)
[shipujin@bogon node]$ 
[shipujin@bogon node]$ uname -a
Linux bogon 6.6.0-0.rc4.20231005git3006adf3be79.36.fc38.loongarch64 #1 SMP PREEMPT Sun Oct  8 14:18:09 CST 2023 loongarch64 GNU/Linux
[shipujin@bogon node]$ 

[shipujin]

(1) How are LoongArch machines provided to nodejs?
I can provide a LoongArch machine, the machine runs in my company's machine room. Later I can provide an ssh access to the LoongArch machine ip, is this allowed?

There is still one question about the way of providing LoongArch machine, look forward to node-loong64 running successfully in the official CI process, thanks~

[targos]

I think access by IP address and SSH is all we need to manage it and connect it to Jenkins.

[targos]

Then it could be part of a nightly job like https://ci.nodejs.org/view/All/job/sxa-rvnodetest/

[shipujin]

I think access by IP address and SSH is all we need to manage it and connect it to Jenkins.

Ok, thank you for your answer.

I will prepare the machine and the user password for accessing the LoongArch machine using ssh in these two days.
Which developer should I send the IP, user and password of the LoongArch machine to? Can I send it to you by email? Then configure "https://ci.nodejs.org/"

Best regards

[mhdawson]

In terms of configuring access we use ssh keys, it would be better if we could send you a public key that you can added to the machine versus a userid/password.

[shipujin]

In terms of configuring access we use ssh keys, it would be better if we could send you a public key that you can added to the machine versus a userid/password.

It is possible to add a public key. I will tell you here after I configure the machine. I can help you put the public key into the nodejs-LoongArch machine.

Best regards

[shipujin]

@nodejs/build @mhdawson @richardlau @sxa @targos @nodejs/platform-loong64
Hi,
I have prepared the LoongArch machine and I am configuring the network.

There are two questions about the way LoongArch machine is provided, hope you can help me, thanks~

1. How is the ip address of the machine provided?
   Is it to send email to send machine ip address

2. What public keys need to be put into the system? Configure build-loong64-CI
   Please send me the download address of the public key, and I will put them into the system for ssh login.

Best regards

[shipujin]

Hello, everyone
I have finished LoongArch machine and network configuration is complete

OS (clfs): Yongbao-2.3-for-nodejs-ci.tar.gz

[shipujin@Sunhaiyong ~]$ 
[shipujin@Sunhaiyong ~]$ uname -a
Linux Sunhaiyong 6.6.0-rc7 #1 SMP PREEMPT Fri Oct 27 01:55:14 UTC 2023 loongarch64 GNU/Linux
[shipujin@Sunhaiyong ~]$ 
[shipujin@Sunhaiyong ~]$ nproc 
32
[shipujin@Sunhaiyong ~]$ free -m
               total        used        free      shared  buff/cache   available
Mem:          130176       24700       98261          23       16378      105476
Swap:              0           0           0
[shipujin@Sunhaiyong ~]$ 
[shipujin@Sunhaiyong ~]$ java --version
openjdk 17.0.8-internal 2023-07-18
OpenJDK Runtime Environment (build 17.0.8-internal+0-adhoc..jdk17-17ugit)
OpenJDK 64-Bit Server VM (build 17.0.8-internal+0-adhoc..jdk17-17ugit, mixed mode)

check nodejs/node 20231101-main

#### main
(1) time make -j32

...

real    17m38.731s                                                  
user    450m48.870s                                                 
sys     16m42.198s

(2) time make test-ci-js 2>&1 | tee ../E.log-nodejs-ci

  ...

Failed tests:
out/Release/node --expose-internals /home/shipujin/node/test/parallel/test-http-correct-hostname.js
make: *** [Makefile:535：test-ci-js] Error 1
                                                                    

real    16m2.169s
user    45m39.270s
sys     9m14.873s

Best regards

Can you guide me on what to do next? Thank you~

@nodejs/build @mhdawson @richardlau @sxa @targos @nodejs/platform-loong64

[shipujin]

Then it could be part of a nightly job like https://ci.nodejs.org/view/All/job/sxa-rvnodetest/

Sorry for my a little too many questions, I would like to ask how "sxa-rvnodetest" accesses CI nodes, thank you

Best regards

[sxa]

I would like to ask how "sxa-rvnodetest" accesses CI nodes

Not sure I understand the question - like all jobs the rv-nodetest one specifies a label for machines to run on under "Restrict where this job can run", and the RISC-V machines that are connected to the CI have that label on it.