the nerd's backyard -- 伪技术杂谈

2022年8月26日星期五

Install GitLab using Docker Compose

1. Install Docker Compose

# 1. remove installed & install dependency
sudo apt remove docker-desktop
sudo rm -r $HOME/.docker/desktop
sudo rm /usr/local/bin/com.docker.cli
sudo apt purge docker-desktop
sudo apt install gnome-terminal

# 2. setup the repository
sudo apt-get update

sudo apt-get install \
    ca-certificates \
    curl \
    gnupg \
    lsb-release

sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo \
    "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
    $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# 3. Install Docker

sudo apt-get update
sudo apt-get install docker-compose-plugin

2. Install GitLab

## 1. Preparation
# Go to the targeted folder
cd /mnt/data/Applications/gitlab

vi /etc/hosts
# private domain to IP
127.0.0.1    gitlab.example.com

vi .env
# set GITLAB_HOME in .env
GITLAB_HOME=/mnt/data/Applications/gitlab

vi docker-compose.yml
# docker-compose.yml
version: '3.6'
services:
  web:
    image: 'gitlab/gitlab-ee:latest'
    restart: always
    hostname: 'gitlab.example.com'
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'http://gitlab.example.com:1080'
        gitlab_rails['gitlab_shell_ssh_port'] = 1022
        # Add any other gitlab.rb configuration here, each on its own line
    ports:
      - '1080:1080'
      - '443:443'
      - '1022:22'
    volumes:
      - '$GITLAB_HOME/:/mnt/data/Applications/gitlab' 
      - '$GITLAB_HOME/config:/mnt/data/Applications/gitlab/config'
      - '$GITLAB_HOME/logs:/mnt/data/Applications/gitlab/logs'
      - '$GITLAB_HOME/data:/mnt/data/Applications/gitlab/data'
    shm_size: '256m'

## 2. docker compose installation
sudo docker --env-file .env up -d

# 3. get password for root
sudo docker exec -it gitlab-web-1 grep 'Password:' /etc/gitlab/initial_root_password

## 4. Browsing GitLab
## http://gitlab.example.com:1080 
## or 
## 192.168.0.xxx:1080

# reconfiguration
sudo docker exec -it gitlab-web-1 editor /etc/gitlab/gitlab.rb
# reconfigure as needed
# ...
sudo docker stop gitlab-web-1
sudo docker restart gitlab-web-1

TBC...

TBC... #TODO SMTP,

2021年11月24日星期三

Mysql on Ubuntu 20.04

## installation

sudo apt install mysql-server

pip install mysql-server

## set up security

sudo mysql_secure_installation

## login

sudo mysql -u root

## create user

CREATE USER 'user'@'localhost' IDENTIFIED BY 'password';

## Create mysql db

## Change mysql==8.0.27 password in Ubuntu 20.04.1

sudo /etc/init.d/mysql stop

(maybe): sudo mkdir /var/run/mysqld && sudo chown mysql /var/run/mysqld

sudo mysqld_safe --user=mysql --skip-grant-tables --skip-networking &

mysql -u root mysql

flush privileges;

ALTER USER 'root'@'localhost' IDENTIFIED BY 'Password=1234';

sudo killall -u mysql

sudo systemctl restart mysql.service

sudo mysql -u root -p

################## Clone DB ################

Make a dump of your source database:
```
mysqldump -uroot -p my_project -r my_project.sql
```
Or, if you only want to dump the database's table structure (schema) without any contents:
```
mysqldump -uroot -p my_project -r my_project.sql --no-data
```
Open up a MySQL shell:
```
mysql -uroot -p
```

From the MySQL shell, create a new database and populate it with the dumped data:

CREATE DATABASE my_project_copy;
USE my_project_copy;
SOURCE my_project.sql;

Create a user and give it permissions to the new database:

CREATE USER new_user IDENTIFIED BY 'some_password';
GRANT ALL ON my_project_copy.* TO 'new_user'@'localhost' IDENTIFIED BY 'some_password';

2019年11月22日星期五

Install TA-Lib on different OS

Ubuntu: #!/usr/bin/env bash
	## sudo apt install build-essential wget -y
	## wget https://artiya4u.keybase.pub/TA-lib/ta-lib-0.4.0-src.tar.gz
	## tar -xvf ta-lib-0.4.0-src.tar.gz
	## cd ta-lib/
	## ./configure --prefix=/usr
	## make
	## sudo make install wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz \ && sudo tar -xzf ta-lib-0.4.0-src.tar.gz \ && sudo rm ta-lib-0.4.0-src.tar.gz \ && cd ta-lib/ \ && sudo ./configure --prefix=/usr \ && sudo make \ && sudo make install \ && cd ~ \ && sudo rm -rf ta-lib/ \ && pip install ta-lib

Mac Os:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" < /dev/null 2> /dev/null
brew install ta-lib


For all:
pip install ta-lib

2018年8月13日星期一

xgboost & lightgbm with gpu on windows

Install latest visual studio and R 3.5.1

XGB: 1. git clone --recursive https://github.com/dmlc/xgboost

2. cmake .. -G"Visual Studio 15 2017 Win64" -T v140 -DUSE_CUDA=ON -DR_LIB=ON

3. cmake --build . --target install --config Release

4. install.packages() with local file

LightGBM:
1. http://lightgbm.readthedocs.io/en/latest/GPU-Windows.html
(address-model=64)
i.e., (CUDA, Rtools, Boost)

2. git clone --recursive https://github.com/Microsoft/LightGBM

cd LightGBM/R-package

3. Set use_gpu to TRUE in R-package/src/install.libs.R to enable the build with GPU support.

4. Rscript build_package.R
R CMD INSTALL lightgbm_2.1.1.tar.gz --no-multiarch

5. Install.packages in R with local file

2017年5月1日星期一

Install GUI of Ubuntu on AWS

1. Connect ssh to ec2 instance.

2. install vncserver:
sudo apt install ubuntu-desktop
sudo apt install vnc4server
sudo apt install gnome-panel

3. Type 'vncserver' to setup pwd

4. Kill vncserver 'vncserver -kill :1'

5. Type following:

vim /home/ubuntu/.vnc/xstartup

and modify

unset SESSION_MANAGER
# exec /etc/.....
gnome-session -session=gnome-classic &
gnome-panel&

('ESC', ':wq', 'ENTER' to quit vim)

Type 'vncserver' to start.

Download TightVNC Viewer to connect.

In AWS website, to set "Inbound rule" to allow connections, with IP::port

Tensorflow with AWS: https://aws.amazon.com/cn/blogs/china/tensorflow-on-aws/

2016年9月26日星期一

Ubuntu install lists

A list of post installation of Ubuntu

sudo apt-get install freeglut3-dev build-essential libx11-dev libxmu-dev libxi-dev libgl1-mesa-glx libglu1-mesa libglu1-mesa-dev

sudo apt-get update

sudo apt-get upgrade

Google Chrome

http://cn.soulmachine.me/2016-08-17-deep-learning-cuda-development-environment/

Driver:

sudo add-apt-repository ppa:graphics-drivers/ppa

sudo apt-get update

sudo apt-get install nvidia-375 (gforce) -377(quadro)

1. 'stop x server'

'Ctrl + Alt + F1' ===> terminal

sudo service lightdm stop

2. download and install driver

sudo sh NVIDIA.......

Cuda  #install

sudo sh cuda_8.0.27_linux.run --tmpdir=/tmp --override

sudo sh cuda_8.0.27.1_linux.run

~/.bashrc

export PATH=/usr/local/cuda-8.0/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

#test

cd ~/NVIDIA_CUDA-8.0_Samples/1_Utilities/deviceQuery

make

./deviceQuery

cd ~/NVIDIA_CUDA-8.0_Samples/5_Simulations/nbody/

make

./nbody -benchmark -numbodies=256000 -device=0

Output:


> Windowed mode

> Simulation data stored in video memory

> Single precision floating point simulation

> 1 Devices used for simulation

gpuDeviceInit() CUDA Device [0]: "GeForce GTX 1080

> Compute 6.1 CUDA device: [GeForce GTX 1080]

number of bodies = 256000

256000 bodies, total time for 10 iterations: 2364.286 ms

= 277.192 billion interactions per second

= 5543.830 single-precision GFLOP/s at 20 flops per interaction

cuDNN

'copy ../cuda/include/*.h  to  /usr/local/cuda-8.0/include/'

'copy ../cuda/lib64/*.* to /usr/local/cuda-8.0/lib64'

**'/usr/local/cuda-8.0' is the path where CUDA installed.

Tensorflow

sudo apt-get install libcurl3-dev

$ conda install virtualenv # suggested by the output of terminal not using pip install
$ conda create --name=tensorflow_env python=3.5
$ source activate tensorflow_env
$ pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl # the code you provided is for the python 3

(to delete env: conda env remove --name tensorflow)

Rebuild tensorflow with CUDA!!!

https://alliseesolutions.wordpress.com/2016/09/08/install-gpu-tensorflow-from-sources-w-ubuntu-16-04-and-cuda-8-0-rc/

Note:

1. Install protobuf 3.0!!MUST BE 3.0

pip install protobuf3

2. MUST downloads tensorflow from online, NOT get from the command line!!!

In build:

$ bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
$ bazel-bin/tensorflow/tools/pip_package/build_pip_package ~/Downloads/tensorflow_pkg
$ sudo pip3 install ~/Downloads/tensorflow_pkg/tensorflow-0.10.0-py3-none-any.whl

######==========================================================================

Caffe:

http://caffe.berkeleyvision.org/installation.html#prerequisites

In compiling opencv, BEFORE 'make':

In file 'modules/cudalegacy/src/graphcuts.cpp' change:

-#if !defined (HAVE_CUDA) || defined (CUDA_DISABLER)

+// GraphCut has been removed in NPP 8.0

+#if !defined (HAVE_CUDA) || defined (CUDA_DISABLER) || (CUDART_VERSION >= 8000)

prereqisites:

1. protobuf

sudo apt-get install autoconf automake libtool curl make g++ unzip

2. glog, etc..

sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev
3. hdf5

sudo apt-get install hdf5-tools

cd /usr/lib/x86_64-linux-gnu

sudo ln -s libhdf5_serial.so.10.1.0 libhdf5.so

sudo ln -s libhdf5_serial_hl.so.10.0.2 libhdf5_hl.so

Changes in Makefile.config

Uncomment 'OPENCV_VERSION := 3', 'USE_CUDNN := 1', 'ANACONDA_HOME := $(HOME)/anaconda3/envs/caffe/', 'PYTHON_INCLUDE := $(ANACONDA_HOME)/include \

  # $(ANACONDA_HOME)/include/python3.5 \

  # $(ANACONDA_HOME)/lib/python3.5/site-packages/numpy/core/include \', 'USE_PKG_CONFIG := 1'

(BEFORE 'make all' for caffe)

find . -type f -exec sed -i -e 's^"hdf5.h"^"hdf5/serial/hdf5.h"^g' -e 's^"hdf5_hl.h"^"hdf5/serial/hdf5_hl.h"^g' '{}' \;

mkdir build
cd build
cmake ..
make all      (Or cmake -DWITH_IPP=ON . && make -j $(nproc) && make install???)
make install
make runtest

C++

CodeBlocks:

sudo add-apt-repository ppa:damien-moore/codeblocks-stable
sudo apt-get update

Anaconda

Pycharm:

sudo add-apt-repository ppa:mystic-mirage/pycharm

sudo apt-get update

sudo apt-get install pycharm-comminity

Notepad++:


sudo add-apt-repository ppa:notepadqq-team/notepadqq

sudo apt-get update

sudo apt-get install notepadqq

订阅：博文 (Atom)