Installing and Configuring XGA

This is a slightly more complex installation than many Python modules, but shouldn’t be too difficult. If you’re having issues feel free to get in contact (see Getting Help section).

Data Required For Using This Module

This is very important - Currently, to make use of this module, you must have access to either cleaned XMM-Newton or eROSITA event lists. For aquiring and cleaning X-ray data, the Python module DAXA is recommended.

Region Files

It will be beneficial if you have region files available, as it will allow XGA to remove interloper sources. If you wish to use existing region files, then they must be in a DS9 compatible format, point sources must be red and extended sources must be green.

The Module

We strongly recommend that you make use of Python virtual environments, or (even better) Conda/Mamba virtual environments when installing XGA.

XGA is available on the popular Python Package Index (PyPI), and can be installed like this:

pip install xga

You can also fetch the current working version from the git repository, and install it:

git clone https://github.com/DavidT3/XGA
cd XGA
python -m pip install .

Alternatively you could use the ‘editable’ option so that any changes you pull from the remote repository are reflected without having to reinstall XGA.

git clone https://github.com/DavidT3/XGA
cd XGA
python -m pip install --editable .

We also provide a Conda lock file in the conda_envs directory (see conda-lock GitHub README on how to install conda-lock), which can be used to create an Anaconda environment with the required dependencies (excepting PyAbel, which has to be installed through pip at this time):

Important

For Python 3.10 and earlier, the optional dependency contextlib-chdir must be installed.

Required Dependencies

XGA depends on some non-Python pieces of software, dependent on the telescope from which your data originates:

For all telescopes:

  • HEASoft’s XSPEC - Version 12.10.1 - It isn’t guaranteed later versions will work.

For XMM data:

  • XMM’s Science Analysis System (SAS) - Version 17.0.0, but other versions should be largely compatible with the software. SAS version 14.0.0 however, does not support features that PSF correction of images depends on.

For eROSITA data:

  • eROSITA Science Analysis Software System (eSASS) - EDR version. It must be installed on the system, we do not support interaction through a Docker container.

All required Python modules can be found in requirements.txt, and should be added to your system during the installation of XGA.

Excellent installation guides for SAS, HEASoft, and eSASS already exist, please see them for detailed installation instructions. XGA will not generate XMM products without detecting SAS, or generate eROSITA products without detecting eSASS, or fit spectra without detecting XSPEC.

Optional Dependencies

XGA can also make use of external software for some limited tasks, but they are not required to use the module as a whole:

  • The R interpreter.

  • Rpy2 - A Python module that provides an interface with the R language in Python.

  • LIRA - An R fitting package.

The R interpreter, Rpy2, and LIRA are all necessary only if you wish to use the LIRA scaling relation fitting function.

Configuring XGA

Before XGA can be used you must fill out a configuration file (a completed example can be found here).

Follow these steps to fill out the configuration file:

  1. Import XGA to generate the initial, incomplete, configuration file. * Note: XGA uses a lazy loading architecture. Importing the module does not perform any filesystem checks or software discovery immediately. These only occur when a configuration variable or data-dependent function is first accessed.

  2. Navigate to ~/.config/xga and open xga.cfg in a text editor. The .config directory is usually hidden, so it is probably easier to navigate via the terminal. * Note: The location of the configuration directory can be overridden by setting the XGA_CONFIG_DIR environment variable. If set, XGA will look for its configuration and store its census files in exactly that directory. Because of XGA’s lazy loading, you can even set this environment variable after importing XGA in a session. * Note: If you need to switch configuration directories mid-session, you can use the reinitialise_xga() function:

    import xga
# Change to a new config directory
xga.reinitialise_xga('/path/to/new/config/')

  3. Take note of the entries that currently have /this/is/required at the beginning, without these entries the module will not function.

  4. Set the directory in which XGA will save the products and files it generates. It is advised to just set it to xga_output, so wherever you run a script that imports XGA it will create a folder called xga_output there. You could choose to use an absolute path and have a global XGA folder however, it wouldn’t make a lot of sense.

  5. You may also set an optional parameter in the [XGA_SETUP] section, ‘num_cores’. If you wish to manually limit the number of cores that XGA is allowed to use, then set this to an integer value, e.g. num_cores = 10. You can also set this at runtime, by importing NUM_CORES from xga and setting that to a value.

  6. The root_<telescope>_dir entry is the path of the parent folder containing all of your observation data for <telescope>. It is not necessary to have data for all telescopes that XGA supports.

  7. Most of the other entries tell XGA how different files are named. clean_pn_evts, for instance, gives the naming convention for the cleaned PN events files that XGA generates products from.

  8. Bear in mind when filling in the file fields that XGA uses the Python string formatting convention, so anywhere you see {obs_id} will be filled formatted with the ObsID of interest when XGA is actually running.

  9. The lo_en and hi_en entries can be used to tell XGA what images and exposure maps you may already have. For instance, if you already had 0.50-2.00keV and 2.00-10.00keV images and exposure maps, you could set lo_en = [‘0.50’, ‘2.00’] and hi_en = [‘2.00’, ‘10.00’].

  10. Finally, the region_file entry tells XGA where region files for each observation are stored (if they exist).

Disclaimer: If region files are supplied for XMM data, XGA also expects at least one image per instrument per observation, for WCS information.

This section aims to be as general as possible, but is biased by how our research group generates and stores our data products. If you are an X-ray astronomer who wishes to use this module, but it seems to be incompatible with your setup, please get in touch or raise an issue.

Remote Data Access: If your data lives on a remote server, and you want to use XGA on a local machine, it is recommended to set up an SFTP connection and mounting the server as an external volume. Then you can fill out the configuration file with paths going through the mount folder. To mount a server, one can follow the steps detailed in this guide.

XGA’s First Run After Configuration

The first time you access any configuration-dependent part of XGA, it will create an ‘observation census’, where it will search through all the observations it can find (based on your entries in the configuration file), check that there are events lists present, and record the pointing RA and DEC. This can take a while, but will only take that long on the first run. This process is fully parallelized and will utilize your available CPU cores to speed up the filesystem I/O.

The module will check the census against your observation directory and see if it needs to be updated on every run.

Census Management

XGA provides the rebuild_census() function for manual management of the observation census. This is useful if you have deleted data from your disk or want to force a fresh scan of your archives.

  • Standard Update: Calling rebuild_census() without arguments is equivalent to the update check performed at startup.

  • Full Rebuild: rebuild_census(full_rebuild=True) deletes the existing census and parses every FITS header from scratch.

  • Cleanup: rebuild_census(clean_dead=True) checks the current census against the filesystem and removes entries for ObsIDs that no longer exist.

import xga
# Prune census entries for data that has been deleted from disk
xga.rebuild_census(clean_dead=True)

Blacklisting ObsIDs

If you don’t wish your analyses to include certain ObsIDs, then you can ‘blacklist’ them and remove them from all consideration, you simply need to add the ObsID to ‘blacklist.csv’, which is located in the same directory as the configuration file. If you need to know where this configuration file is located, import CONFIG_FILE from xga.utils.

It is possible that you might want to do this so that ObsIDs with significant problems (flaring, for instance), don’t contribute to and spoil your current analysis.