CARLA<\/strong> has been developed from the ground up to support development, training, and validation of autonomous driving systems. In addition to open-source code and protocols, CARLA provides open digital assets (urban layouts, buildings, vehicles) that were created for this purpose and can be used freely. The simulation platform supports flexible specification of sensor suites, environmental conditions, full control of all static and dynamic actors, maps generation and much more.<\/p>\n Though CARLA is open source and freely distributed under MIT License, the license of Unreal Engine<\/a> that CARLA uses is Source-available commercial software with royalty model for commercial use.<\/em><\/p>\n CARLA and Unreal Engine in CSF3 is to be used for academic and research purposes only. They must not be used for commercial purposes.<\/strong><\/p>\n Further license information can be seen at the following links:<\/p>\n To get access to CARLA on the CSF3, you must request it via our form help form<\/a> and confirm that your usage will comply with the terms and licenses above. Please note that we may ask you for some further info to support your request.<\/p>\n To access the software you must load ONE of the modulefiles:<\/p>\n Please do NOT run CARLA on the login nodes. CARLA requires a lot of resources and can overload the login nodes. Normally, CARLA should be run via batch. However, once-in-a-while some user might need to visualize the scenes they are working on and for this purpose please use a interactive job<\/a> (details below<\/a>).<\/p>\n The steps for running CARLA in Graphical mode in an interactive session are:<\/p>\n The set of commands needed to accomplish this are:<\/p>\n This should open a new window with the default scene. Please note that it will take some time to load for the first time. Initially it will be all dark and seem as if nothing is happening. Just give it some time. After a few seconds the default scene should come up in the window.<\/p>\n Please note that the frame rate will be low since the rendering is happening over network remotely, not to a monitor directly connected to the hardware where it is running. At this stage you will be able to navigate the scene with the keys Q\/E\/W\/S\/A\/D<\/strong>.<\/p>\n \ud83d\udcdd See Manipulation using API<\/a> section below for interacting with this CARLA server instance.<\/p>\n Create a batch submission script like the following and submit a self-contained job to the batch.<\/p>\n Submit the jobscript using:<\/p>\n where scriptname<\/em> is the name of your jobscript.<\/p>\n You can manipulate CARLA server process running in the compute node through its Python API from the login node itself. Submit the jobscript using:<\/p>\n where scriptname<\/em> is the name of your jobscript.<\/p>\n <\/a> Once the job has started, you can manipulate CARLA server process running in the compute node using its Python API from the login node itself. Note down the Next run the following commands in login node to run Python and manipulate CARLA server running in a compute node via its API:<\/p>\n <\/p>\n Please do NOT run CARLA on the login nodes. CARLA requires a lot of resources and can overload the login nodes. Normally, CARLA should be run via batch. However, once-in-a-while some user might need to visualize the scenes they are working on and for this purpose please use a interactive job<\/a> (details below<\/a>).<\/p>\n The steps for running CARLA in Graphical mode in an interactive session are:<\/p>\n The set of commands needed to accomplish this are:<\/p>\n This should open a new window with the default scene. Please note that it will take some time to load for the first time. Initially it will be all dark and seem as if nothing is happening. Just give it some time. After a few seconds the default scene should come up in the window. <\/p>\n Please note that the frame rate will be low since the rendering is happening over network remotely, not to a monitor directly connected to the hardware where it is running. At this stage you will be able to navigate the scene with the keys Q\/E\/W\/S\/A\/D<\/strong>.<\/p>\n \ud83d\udcdd See Manipulation using API<\/a> section below for interacting with this CARLA server instance.<\/p>\n Create a batch submission script like the following and submit a self-contained job to the batch.<\/p>\n Submit the jobscript using:<\/p>\n where scriptname<\/em> is the name of your jobscript.<\/p>\n You can manipulate CARLA server process running in the compute node through its Python API from the login node itself. Submit the jobscript using:<\/p>\n where scriptname<\/em> is the name of your jobscript.<\/p>\n <\/a> Once the job has started, you can manipulate CARLA server process running in the compute node using its Python API from the login node itself. Note down the Next run the following commands in login node to run Python and manipulate CARLA server running in a compute node via its API:<\/p>\n <\/p>\n If you have a handy tip you would like to share, please contact us<\/a>.<\/p>\n By default, the CARLA server process running in the compute node runs\/listens on ports:<\/p>\n 2000<\/strong>, 2001<\/strong> and 2002<\/strong> in version 0.10.0 You will normally not need to change the port numbers. However, with increasing number of users of CARLA it is possible that your interactive session request lands you on a compute node where a CARLA job is already running and using that port because up to 4 GPU jobs can run on a single GPU node. This can prevent your CARLA process from running when your run CarlaUnreal.sh \/ CarlaUE4.sh<\/strong> (or carla) and throw following error message for CARL 0.10.0:<\/p>\n libc++abi: terminating due to uncaught exception of type std::__1::system_error: bind: Address already in use<\/strong><\/span><\/p>\n and following error message for CARL 0.9.10.1:<\/p>\n Exception thrown: bind: Address already in use<\/strong><\/span><\/p>\n In such circumstances you can manually change the port CARLA will use by adding the command-line argument: Try using port number (N<\/span><\/strong>) 15000<\/strong> and above as these are safe port numbers to use since they are generally not used by other processes.<\/p>\n The second and third port will be automatically set to N+1<\/strong> and N+2<\/strong> in case of CARLA version 0.10.0 Don’t forget to use that same port number in the API command: While launching the CARLA simulator in interactive graphical mode, you can control the window size with Downloaded Dataset generated via privileged agent – autopilot (\/team_code_autopilot\/autopilot.py) is available in CSF3 and can be accessed after loading the CARLA 0.10.0 with Unreal Engine 5 Documentation<\/a> None.<\/p>\n","protected":false},"excerpt":{"rendered":" Overview CARLA has been developed from the ground up to support development, training, and validation of autonomous driving systems. In addition to open-source code and protocols, CARLA provides open digital assets (urban layouts, buildings, vehicles) that were created for this purpose and can be used freely. The simulation platform supports flexible specification of sensor suites, environmental conditions, full control of all static and dynamic actors, maps generation and much more. Restrictions on use Though CARLA.. Read more »<\/a><\/p>\n","protected":false},"author":21,"featured_media":0,"parent":86,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-10457","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/10457","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/users\/21"}],"replies":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/comments?post=10457"}],"version-history":[{"count":22,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/10457\/revisions"}],"predecessor-version":[{"id":10680,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/10457\/revisions\/10680"}],"up":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/86"}],"wp:attachment":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/media?parent=10457"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Restrictions on use<\/h2>\n
\n
Set up procedure<\/h2>\n
carla<\/code> group membership.<\/div>\n\r\nmodule load apps\/python\/carla\/0.10.0<\/strong> #CARLA v0.10.0 with Unreal Engine 5\r\nmodule load apps\/python\/carla\/2022<\/strong> #CARLA Transfuser 2022-tree (v0.9.10.1) with Unreal Engine 4\r\n<\/pre>\n
Running the application – CARLA v0.10.0 with Unreal Engine 5<\/h2>\n
\nJobs should be submitted to the compute nodes via batch (example below<\/a>).<\/p>\nInteractive job example<\/h3>\n
\n
# Start an interactive session with 1 V100 GPU, 1 CPU core for 1 Hour\r\n# This should land you in an interactive node if resources are available.\r\n# If you do not get an interactive session, try again later\r\nsrun-x11 -p gpuV -G 1 -n 1 -t 0-1 --pty bash<\/strong>\r\n\r\n# Once you have landed in an interactive node load the CARLA module\r\nmodule purge\r\nmodule load apps\/python\/carla\/0.10.0<\/strong>\r\n\r\n# Launch CARLA interactive GUI\r\nCarlaUnreal.sh -nosound<\/strong> #or just: carla -nosound<\/strong>\r\n<\/pre>\n
\nlibc++abi: terminating due to uncaught exception of type std::__1::system_error: bind: Address already in use<\/strong><\/span>
\nSee tip on how to change port number<\/a> below to learn how to overcome this error.<\/div>\n
\n\ud83d\udcdd See tip on controlling the resolution<\/a> below for slightly better frame rate.<\/p>\n\n
Batch job submission<\/h3>\n
#!\/bin\/bash --login\r\n#SBATCH -p gpuV # v100 GPU. Other available types- gpuA(A100), gpuL(L40)\r\n#SBATCH -G 1 # 1 GPU\r\n#SBATCH -t 1-0 # Wallclock limit (1-0 is 1-day & 0-hour, 4-0 is max permitted)\r\n#SBATCH -n 8 # Select the no. of CPU cores\r\n # Can use up to 8 CPUs with an v100 GPU.\r\n # Can use up to 12 CPUs with an A100 GPU.\r\n # Can use up to 12 CPUs with an L40s GPU.\r\n#SBATCH -J carla\t # Jobname\r\n#SBATCH -o %x.o%j\t # %x = SLURM_JOB_NAME\r\n#SBATCH -e %x.e%j\t # %j = SLURM_JOB_ID\r\n\r\n# Load the module\r\nmodule purge\r\nmodule load apps\/python\/carla\/0.10.0\r\n\r\n# Find free port to run CARLA server process\r\nexport FREEPORT=$(find-freeport)\r\n\r\n# Optional: Save the portnumber to a text file, for use by your python code later.\r\necho $FREEPORT > FREEPORT.txt\r\n\r\n# Launch CRALA server (notice the & at the end of this line)\r\nCarlaUnreal.sh -carla-port=$FREEPORT -RenderOffScreen -nosound &\r\n\r\n# Run your Python script\r\npython myscript.py\r\n #\r\n # If you need to know the port number in your python code, you can\r\n # read the FREEPORT.txt file (see above), or read the environment variable:\r\n # import os;\r\n # portnum=os.getenv(\"FREEPORT\")\r\n # Use 127.0.0.1 for the hostname.\r\n<\/pre>\n
sbatch scriptname<\/em><\/pre>\n
Manipulating CARLA server process remotely from login node through its Python API<\/h3>\n
\nFor this, first create a batch submission script like the following and submit it to the batch.<\/p>\n#!\/bin\/bash --login\r\n#SBATCH -p gpuV # v100 GPU. Other available types- gpuA(A100), gpuL(L40)\r\n#SBATCH -G 1 # 1 GPU\r\n#SBATCH -t 1-0 # Wallclock limit (1-0 is 1-day & 0-hour, 4-0 is max permitted)\r\n#SBATCH -n 8 # Select the no. of CPU cores\r\n # Can use up to 8 CPUs with an v100 GPU.\r\n # Can use up to 12 CPUs with an A100 GPU.\r\n # Can use up to 12 CPUs with an L40s GPU.\r\n#SBATCH -J carla\t # Jobname\r\n#SBATCH -o %x.o%j\t # %x = SLURM_JOB_NAME\r\n#SBATCH -e %x.e%j\t # %j = SLURM_JOB_ID\r\n\r\n# Load the module\r\nmodule purge\r\nmodule load apps\/python\/carla\/0.10.0\r\n\r\n# Find free port to run CARLA server process\r\nexport FREEPORT=$(find-freeport)\r\n\r\necho \"Your CARLA job $SLURM_JOB_ID is running in host: $HOSTNAME\"\r\necho \"CARLA is available in port number: $FREEPORT\"\r\necho \"Job is using $SLURM_GPUS GPU(s) with ID(s) $CUDA_VISIBLE_DEVICES and $SLURM_NTASKS CPU core(s)\"\r\n\r\nCarlaUnreal.sh -carla-port=$FREEPORT -RenderOffScreen -nosound\r\n\r\n# Comment the above line and uncomment the below line if you also want to watch the scene in real time\r\n#CarlaUnreal.sh -carla-port=$FREEPORT -nosound\r\n<\/pre>\n
sbatch scriptname<\/em><\/pre>\n
\nWait for the job to start. Monitor your queue using the command: squeue<\/strong><\/a>.<\/p>\n
\nFor this you will need the hostname<\/code><\/strong> where the job is running and the port number<\/code><\/strong> once the batch job has started.
\nRun the following command to view the hostname and the port number after the job has started running:<\/p>\nhead -n2 <jobname>.o<jobid> <\/strong> # Replace the <jobname><\/strong> and <jobid><\/strong> with the real ones.\r\n # You need to set the #SBATCH -J, -o and -e options \r\n # in your jobscript as shown above.\r\n<\/pre>\n
hostname<\/code> where the job is running and the port number<\/code>.<\/p>\nmodule purge\r\nmodule load apps\/python\/carla\/0.10.0\r\n\r\npython<\/strong>\r\n>>>import carla<\/strong>\r\n>>>client = carla.Client('<hostname>', <port_number>)<\/strong> \r\n# Replace <hostname> and <port_number> with the real hostname and port number obtained in previous step\r\n# E.g.: client = carla.Client('node804.csf3.man.alces.network', 2000)<\/span>\r\n\r\n# You can then do things like:\r\n>>>world = client.get_world()<\/strong>\r\n>>>print(world)<\/strong>\r\nWorld(id=11172095543979033550)<\/strong><\/span>\r\n\r\n>>>level = world.get_map()<\/strong>\r\n>>>print(level)<\/strong>\r\nMap(name=Carla\/Maps\/Town10HD_Opt)<\/strong><\/span>\r\n\r\n>>>print(client.get_available_maps())<\/strong>\r\n['\/Game\/Carla\/Maps\/Mine_01', '\/Game\/Carla\/Maps\/Town10HD_Opt']<\/strong><\/span>\r\n\r\n>>>world = client.load_world('\/Game\/Carla\/Maps\/Mine_01')<\/strong>\r\n>>>level = world.get_map()<\/strong>\r\n>>>print(level)<\/strong>\r\nMap(name=Carla\/Maps\/Mine_01)<\/strong><\/span>\r\n<\/pre>\nRunning the application – CARLA Transfuser 2022-tree (v0.9.10.1) with Unreal Engine 4<\/h2>\n
\nJobs should be submitted to the compute nodes via batch (example below<\/a>).<\/p>\nInteractive job example<\/h3>\n
\n
# Start an interactive session with 1 V100 GPU, 1 CPU core for 1 Hour\r\n# This should land you in an interactive node if resources are available.\r\n# If you do not get an interactive session, try again later\r\n# Rendering of the scene is better with V100. Scenes rendered with A100 has some visual artefacts.\r\nsrun-x11 -p gpuV -G 1 -n 1 -t 0-1 --pty bash<\/strong>\r\n\r\n# Once you have landed in an interactive node load the CARLA module\r\nmodule purge\r\nmodule load apps\/python\/carla\/2022<\/strong>\r\n\r\n# Launch CARLA interactive GUI\r\nCarlaUE4.sh <\/strong> #or just: carla <\/strong>\r\n<\/pre>\n
\nException thrown: bind: Address already in use<\/strong><\/span>
\nSee tip on how to change port number<\/a> below to learn how to overcome this error.<\/div>\n
\n\ud83d\udcdd See tip on controlling the resolution<\/a> below for slightly better frame rate.<\/p>\n\n
Batch job submission<\/h3>\n
#!\/bin\/bash --login\r\n#SBATCH -p gpuV # v100 GPU. Other available types- gpuA(A100), gpuL(L40)\r\n#SBATCH -G 1 # 1 GPU\r\n#SBATCH -t 1-0 # Wallclock limit (1-0 is 1-day & 0-hour, 4-0 is max permitted)\r\n#SBATCH -n 8 # Select the no. of CPU cores\r\n # Can use up to 8 CPUs with an v100 GPU.\r\n # Can use up to 12 CPUs with an A100 GPU.\r\n # Can use up to 12 CPUs with an L40s GPU.\r\n#SBATCH -J carla\t # Jobname\r\n#SBATCH -o %x.o%j\t # %x = SLURM_JOB_NAME\r\n#SBATCH -e %x.e%j\t # %j = SLURM_JOB_ID\r\n\r\n# Load the module\r\nmodule purge\r\nmodule load apps\/python\/carla\/2022\r\n\r\n# Find free port to run CARLA server process\r\nexport FREEPORT=$(find-freeport)\r\n\r\n# Optional: Save the portnumber to a text file, for use by your python code later.\r\necho $FREEPORT > FREEPORT.txt\r\n\r\n# Launch CRALA server (notice the & at the end of this line)\r\n# This is the method to make CARLA v0.9.10 run in off-screen mode\r\nDISPLAY= CarlaUE4.sh -carla-rpc-port=$FREEPORT -opengl &\r\n\r\n# Run your Python script\r\npython myscript.py\r\n #\r\n # If you need to know the port number in your python code, you can\r\n # read the FREEPORT.txt file (see above), or read the environment variable:\r\n # import os;\r\n # portnum=os.getenv(\"FREEPORT\")\r\n # Use 127.0.0.1 for the hostname.\r\n<\/pre>\n
sbatch scriptname<\/em><\/pre>\n
Manipulating CARLA server process remotely from login node through its Python API<\/h3>\n
\nFor this, first create a batch submission script like the following and submit it to the batch.<\/p>\n#!\/bin\/bash --login\r\n#SBATCH -p gpuV # v100 GPU. Other available types- gpuA(A100), gpuL(L40)\r\n#SBATCH -G 1 # 1 GPU\r\n#SBATCH -t 1-0 # Wallclock limit (1-0 is 1-day & 0-hour, 4-0 is max permitted)\r\n#SBATCH -n 8 # Select the no. of CPU cores\r\n # Can use up to 8 CPUs with an v100 GPU.\r\n # Can use up to 12 CPUs with an A100 GPU.\r\n # Can use up to 12 CPUs with an L40s GPU.\r\n#SBATCH -J carla\t # Jobname\r\n#SBATCH -o %x.o%j\t # %x = SLURM_JOB_NAME\r\n#SBATCH -e %x.e%j\t # %j = SLURM_JOB_ID\r\n\r\n# Load the module\r\nmodule purge\r\nmodule load apps\/python\/carla\/2022\r\n\r\n# Find free port to run CARLA server process\r\nexport FREEPORT=$(find-freeport)\r\n\r\necho \"Your CARLA job $SLURM_JOB_ID is running in host: $HOSTNAME\"\r\necho \"CARLA is available in port number: $FREEPORT\"\r\necho \"Job is using $SLURM_GPUS GPU(s) with ID(s) $CUDA_VISIBLE_DEVICES and $SLURM_NTASKS CPU core(s)\"\r\n\r\n# This is the method to make CARLA v0.9.10 run in off-screen mode\r\nDISPLAY= CarlaUE4.sh -carla-rpc-port=$FREEPORT -opengl\r\n\r\n# Comment the above line and uncomment the below line if you also want to watch the scene in real time\r\n#CarlaUE4.sh -carla-rpc-port=$FREEPORT\r\n<\/pre>\n
sbatch scriptname<\/em><\/pre>\n
\nWait for the job to start. Monitor your queue using the command: squeue<\/strong><\/a>.<\/p>\n
\nFor this you will need the hostname<\/code><\/strong> where the job is running and the port number<\/code><\/strong> once the batch job has started.
\nRun the following command to view the hostname and the port number after the job has started running:<\/p>\nhead -n2 <jobname>.o<jobid> <\/strong> # Replace the <jobname><\/strong> and <jobid><\/strong> with the real ones.\r\n # You need to set the #SBATCH -J, -o and -e options \r\n # in your jobscript as shown above.\r\n<\/pre>\n
hostname<\/code> where the job is running and the port number<\/code>.<\/p>\nmodule purge\r\nmodule load apps\/python\/carla\/2022\r\n\r\npython<\/strong>\r\n>>>import carla<\/strong>\r\n>>>client = carla.Client('<hostname>', <port_number>)<\/strong> \r\n# Replace <hostname> and <port_number> with the real hostname and port number obtained in previous step\r\n# E.g.: client = carla.Client('node805.csf3.man.alces.network', 2000)<\/span>\r\n\r\n# You can then do things like:\r\n>>>world = client.get_world()<\/strong>\r\n>>>print(world)<\/strong>\r\nWorld(id=16098500494432822930)<\/strong><\/span>\r\n\r\n>>>level = world.get_map()<\/strong>\r\n>>>print(level)<\/strong>\r\nMap(name=Town03)<\/strong><\/span>\r\n\r\n>>>print(client.get_available_maps())<\/strong>\r\n\r\n['\/Game\/Carla\/Maps\/Town01', '\/Game\/Carla\/Maps\/Town06', '\/Game\/Carla\/Maps\/Town02', \r\n'\/Game\/Carla\/Maps\/Town07', '\/Game\/Carla\/Maps\/Town03', '\/Game\/Carla\/Maps\/Town10HD', \r\n'\/Game\/Carla\/Maps\/Town04', '\/Game\/Carla\/Maps\/Town05']\r\n<\/strong><\/span>\r\n\r\n>>>world = client.load_world('\/Game\/Carla\/Maps\/Town01')<\/strong>\r\n>>>level = world.get_map()<\/strong>\r\n>>>print(level)<\/strong>\r\nMap(name=Town01)<\/strong><\/span>\r\n<\/pre>\nAdditional useful information and Tips<\/h2>\n
Ports used by CARLA server process<\/h3>\n
\nand
\n2000<\/strong> and 2001<\/strong> in older versions (0.9.10.1)<\/p>\n
\n-carla-port=N<\/span><\/code><\/strong> to the CarlaUnreal.sh \/ CarlaUE4.sh<\/strong> (or carla) command.<\/p>\nE.g.:\r\nCarlaUnreal.sh -nosound -carla-port=15000\r\nor\r\nCarlaUE4.sh -carla-port=15000\r\n<\/strong><\/pre>\n
\nThe second port will be automatically set to N+1<\/strong> in case of CARLA version 0.9.10.1<\/p>\n
\nclient = carla.Client('<hostname>', <port_number>)<\/strong><\/code><\/p>\nControlling the resolution<\/h3>\n
-ResX=N<\/code><\/strong> and -ResY=N<\/code><\/strong> argument to the CarlaUnreal.sh \/ CarlaUE4.sh<\/strong> (or carla) command.<\/p>\nCarlaUnreal.sh -ResX=N<\/em> -ResY=M<\/em> other flags...<\/em>\r\n<\/pre>\n
Dataset<\/h3>\n
apps\/python\/carla\/2022<\/code><\/strong> module:<\/p>\n\r\nmodule purge\r\nmodule load apps\/python\/carla\/2022\r\nls -l $CARLAROOT\/data\/\r\n<\/pre>\n
Further info<\/h2>\n
\nCARLA 0.9.10 with Unreal Engine 4 Documentation<\/a><\/p>\nUpdates<\/h2>\n