{"id":4045,"date":"2017-07-25T14:18:57","date_gmt":"2017-07-25T14:18:57","guid":{"rendered":"http:\/\/ri.itservices.manchester.ac.uk\/csf-apps\/?page_id=4045"},"modified":"2017-07-26T08:31:15","modified_gmt":"2017-07-26T08:31:15","slug":"bgen","status":"publish","type":"page","link":"https:\/\/ri.itservices.manchester.ac.uk\/csf-apps\/software\/applications\/bgen\/","title":{"rendered":"BGEN"},"content":{"rendered":"

Overview<\/h2>\n

BGEN<\/a> provides utility programs to process files in the BGEN format<\/a>.<\/p>\n

These tools will be of interest to researchers working with the UK BioBank full-release<\/a> dataset.<\/p>\n

The master<\/em> branch from the repository (commit id d1f03a2c308a, downloaded 24-July-2017) is installed on the CSF.<\/p>\n

Restrictions on use<\/h2>\n

This version is released under the Boost Software License v1.0<\/a> – there are no restrictions on access on the CSF.<\/p>\n

Set up procedure<\/h2>\n

To access the software you must first load the modulefile:<\/p>\n

\r\nmodule load apps\/gcc\/bgen\/latest\r\n\r\n# The latest<\/em> modulefile currently give you:\r\n# apps\/gcc\/bgen\/d1f03a2c308a\r\n<\/pre>\n
The following tools are available:<\/p>\n
\r\nbgenix\r\ncat-bgen\r\nedit-bgen\r\n<\/pre>\n
You can add the -help<\/code> flag to each tool to see the command-line flags (you may do this on the login node). For example:<\/p>\n
\r\nbgenix -help\r\n<\/pre>\n
You may also wish to load the utility modulefile:<\/p>\n
\r\nmodule load tools\/env\/ukbiobank-full-release\r\n<\/pre>\n
This will set some environment variables<\/a> used in the examples below to make accessing the folder where the UK Bio Bank datasets are kept a little easier.<\/p>\n
Running the application<\/h2>\n
Please do not run BGEN tools on the login node. Jobs should be submitted to the compute nodes via batch.<\/p>\n
Serial batch job submission<\/h3>\n
Make sure you have the modulefile loaded then create a batch submission script, for example:<\/p>\n
\r\n#!\/bin\/bash\r\n#$ -cwd             # Job will run from the current directory\r\n#$ -V               # Inherit settings from modulefile loaded on login node\r\n\r\nbgenix -g path\/to\/filename.bgen<\/em> arg2<\/em> arg3<\/em> ... \r\n<\/pre>\n
Submit the jobscript using: <\/p>\n
qsub scriptname<\/em><\/pre>\n
where scriptname<\/em> is the name of your jobscript.<\/p>\n
Working with the UK BioBank Full Release Dataset<\/h2>\n
The BGEN website offers some advice<\/a> on using bgenix<\/code> with the UK BioBank data. In summary, there is a problem in that the UK BioBank filenames do not<\/em> match the expected format used by bgenix<\/code>.<\/p>\n
For example, the EGAD00010001225\/001\/ dataset contains files of the form:<\/p>\n
\r\nBioBank BGEN filename    BioBank INDEX filename     INDEX filename expected by bgenix\r\n---------------------    ----------------------     ---------------------------------\r\nukb_imp_chr1_v2.bgen     ukb_bgi_chr1_v2.bgi        ukb_imp_chr1_v2.bgen.bgi\r\nukb_imp_chr2_v2.bgen     ukb_bgi_chr2_v2.bgi        ukb_imp_chr2_v2.bgen.bgi\r\n...                      ...                        ...\r\nukb_imp<\/strong>_chrN<\/em>_v2.bgen<\/strong>     ukb_bgi<\/strong>_chrN<\/em>_v2.bgi<\/strong>        ukb_imp<\/strong>_chrN<\/em>_v2.bgen.bgi<\/strong>\r\n<\/pre>\n
If you simply run:<\/p>\n
\r\nbgenix -g ukb_imp_chr1_v2.bgen -list\r\n  #\r\n  # Let bgenix generate the index filename based\r\n  # on the name of the input bgen filename.\r\n  # THIS WILL FAIL!\r\n<\/pre>\n
It will try to find an index file named ukb_imp_chr1_v2.bgen.bgi<\/code> but this is incorrect for the UK BioBank dataset. You will receive an error message:<\/p>\n
\r\n!! Error opening index file \"ukb_imp_chr1_v2.bgen.bgi\":\r\nCould not open the index file \"ukb_imp_chr1_v2.bgen.bgi\"\r\n<\/pre>\n
It is possible to add the -i<\/code> flag to the bgenix<\/code> command-line to specify explicitly a different index filename use. For example:<\/p>\n
\r\nbgenix -g ukb_imp_chr1_v2.bgen -i ukb_bgi_chr1_v2.bgi<\/strong> -list\r\n  #\r\n  # Tell bgenix what index filename to use.\r\n  # THIS WILL SUCCEED!\r\n<\/pre>\n
If you wish to script the generation of index filenames from bgen filenames inside a jobscript you can use commands such as:<\/p>\n
\r\nBGENFILE=ukb_imp_chr1_v2.bgen\r\nINDEXFILE=`echo $BGENFILE | sed 's\/imp\/bgi\/g;s\/bgen\/bgi\/g'`\r\nbgenix -g $BGENFILE -i $INDEXFILE args...<\/em>\r\n<\/pre>\n
Job Array Example<\/h3>\n
The following job array<\/a> will process all of the files:<\/p>\n
\r\nukb_imp_chr1_v2.bgen\r\nukb_imp_chr2_v2.bgen\r\n...\r\nukb_imp_chr22_v2.bgen\r\n<\/pre>\n
Create a batch submission script similar to:<\/p>\n
\r\n#!\/bin\/bash --login\r\n#$ -cwd\r\n### Note we will load the modulefiles in the jobscript hence\r\n### no '#$ -V' line and we've added --login above.\r\n\r\n### Automatically run 22 copies of this job (each uses 1 core)\r\n#$ -t 1-22\r\n\r\n# We load the modulefiles in the jobscript (hence no #$ -V line)\r\nmodule load apps\/gcc\/bgen\/latest\r\nmodule load tools\/env\/ukbiobank-full-release\r\n\r\n### ${SGE_TASK_ID} is automatically replaced by the number 1, 2, 3, ..., 22 \r\nBGENFILE=${UKBB_IMPUTATION_DIR}\/ukb_imp_chr${SGE_TASK_ID}_v2.bgen\r\n\r\n### Generate the correct index filename for bgenix\r\nINDEXFILE=`echo $BGENFILE | sed 's\/imp\/bgi\/g;s\/bgen\/bgi\/g'`\r\n\r\n### Run bgenix\r\nbgenix -g $BGENFILE -i $INDEXFILE -list\r\n<\/pre>\n
Submit the jobscript using: <\/p>\n
qsub scriptname<\/em><\/pre>\n
where scriptname<\/em> is the name of your jobscript.<\/p>\n
Further info<\/h2>\n