LAPIS

common-operations

1 min read

Some frequently used operations, listed here for convenience:

Adding SSH Key

(generate a key):

ssh-keygen -t ed25519

add pubkey to remote machine:

echo "your-ssh-key" >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys

adding a private key:

ssh-add your_file.pem

Disabling password login

only allowing login via authorized ssh keys:

nano /etc/ssh/sshd_config
 
# PasswordAuthentication no
# ChallengeResponseAuthentication no   <- if exists, modify this
# UsePAM no
 
# systemctl restart sshd  <- use below if not working
service ssh restart

Sync files to s3 every 2 minutes

one method:

watch -n 120 ~/s5cmd sync . "s3://dataset-ingested/datagen_workspace/02_nai_default/"

another way:

# 自动定时上传训练的到s3
while true; do aws s3 sync ./ s3://bucket-external/model_store/fulldan_artstation_600k_test/  && sleep 900; done
 
# 在aws机器上定时同步文件
while true; do aws s3 sync s3://bucket-external/model_store/fulldan_artstation_600k_test/ ./  && sleep 900; done