Program Files
There are a series of scripts used to run DENSS. The core functions of DENSS are stored in the saxstats.py module. The main script to run a single reconstruction is denss.py. There are also scripts for performing multiple DENSS runs and aligning and averaging maps, as well as helper scripts that can perform some useful tasks. The following is a full list of the available scripts and a brief description of what they can do:
- denss.py – A tool for calculating an electron density map from solution scattering data.
- denss.fit_data.py – A tool for fitting solution scattering data with smooth function based on Moore’s algorithm for fitting a trigonometric series.
- denss.all.py – Generate, align, and average multiple electron density maps using DENSS.
- superdenss – Generate, align and average multiple electron density maps using EMAN2.
- denss.align.py – A tool for aligning electron density maps.
- denss.align2xyz.py – A tool for aligning an electron density map such that its principal axes of inertia are aligned with the x,y,z axes.
- denss.align_by_principal_axes.py – A tool for aligning an electron density map to another electron density map based only on alignment of principal axes (no minimization).
- denss.average.py – A tool for averaging multiple pre-aligned electron density maps.
- denss.align_and_average.py – A tool for aligning and averaging multiple electron density maps.
- denss.refine.py – A tool for refining an electron density map from solution scattering data.
- denss.calcfsc.py – A tool for calculating the Fourier Shell Correlation between two pre-aligned MRC formatted electron density maps.
- denss.get_info.py – Print some basic information about an MRC file.
- denss.pdb2mrc.py – A tool for calculating electron density maps from pdb files.
- denss.mrc2sas.py – A tool for calculating simple scattering profiles from MRC formatted electron density maps.
- denss.regrid.py – A tool for regridding a scattering profile to another set of q values.
- denss.mrcops.py – A tool for performing basic operations on MRC formatted electron density maps.
- denss.select_enantiomer.py – A tool for selecting the best enantiomer of a density map.
- denss.select_enantiomers.py – A tool for selecting the best enantiomers from a set of multiple density maps.
- denss.generate_reference.py – A tool for generating a reference from a set of maps using a binary averaging procedure.
- best_enantiomers.sh – A tool for generating and selecting enantiomers using EMAN2.
- fsc2res.py – A tool for plotting Fourier Shell Correlation curves and estimating resolution.
Input Data
Solution scattering data, in particular small angle X-ray scattering (SAXS) data, are typically highly oversampled, meaning that there are far more data points measured in an experiment than there are actual unique pieces of information available. SAXS data are often 20 to 50-fold oversampled, depending on the size of your particle and the geometry of your experiment.
However, running DENSS at such high oversampling ratios is computationally costly, as the size of the array grows as N3. As DENSS typically runs at an oversampling ratio of 3 to 5 (see below for details), we need to first reduce the highly oversampled SAXS data to something more manageable, while still retaining all the useful precision brought about by the experimental oversampling.
To do this, we utilize well-known procedures for performing an indirect Fourier transform of the SAXS data. Usually this is used for extracting pair distribution functions, here we need it for providing a smooth fit to the experimental scattering data and for estimating I(0), which is an important parameter in the data.
Multiple file formats are currently acceptable to denss.py and denss.all.py (the two primary scripts for running DENSS): .dat files, .fit files or .out files. Smooth fits will be extracted from .fit and .out files. Raw data can be given as a .dat file, which will then be automatically fit with a smooth curve. Alternatively you can manually run the fitting with the denss.fit_data.py script below. Files ending in .dat that are already smooth curves can also be used, as DENSS will check if the file is raw data or a smooth fit.
Using denss.fit_data.py
A script called denss.fit_data.py is provided which can be used to fit experimental data with a smooth curve based on an extended version of Peter Moore’s approach (Moore 1979) using a trigonometric series. The denss.fit_data.py script includes a simple interactive GUI for selecting Dmax and the smoothing factor alpha and displays the experimental data, the smooth fit to the data, and the real space pair distribution function. denss.fit_data.py will save a .fit file containing the smooth fit to the data which can then be used as input to denss.py (see below). Additionally, useful parameters calculated from the fit, such as the radius of gyration and Porod volume, are displayed and saved in the file header. The manuscript describing the mathematical derivation and the algorithm of this new approach is currently in preparation.
denss.fit_data.py can be run simply from the command line as:
> denss.fit_data.py -f experimental_data.dat
where experimental_data.dat is the noisy scattering profile, given as a three-column ASCII text file with columns q, I, error. An interactive GUI will appear showing the experimental scattering profile on the left along with the fit to the data, and the associated pair distribution function (P(r)) on the right. Two interactive sliders on the bottom left can be adjusted for Dmax (the maximum particle dimension) and the alpha smoothing factor. See denss.fit_data.py -h for more options. When opened, denss.fit_data.py will estimate Dmax directly from the data and will estimate the optimal alpha value to maximize the smoothness of the P(r) curve while still fitting the I(q) profile. The user can also trim the beginning and ending data points by entering values in the “First point” and “Last point” input boxes. Additionally, by default the algorithm will extrapolate the fitted intensity curve to ameliorate truncation ripples in the P(r) curve. The “Extrapolate” checkbox can be clicked to disable this option if desired. The size parameters calculated from the fitted curve (including the forward scattering I(0), radius of gyration Rg, average length vector r, Porod volume Vp, Volume of correlation Vc, and length of correlation lc) will be displayed with their corresponding uncertainties below the P(r) plot.
When closing the GUI window, this will save a *.fit file that can be used with denss.py for density reconstruction. Additional details such as the Dmax will be printed to the console (Dmax will be needed for denss.py also). The *.fit file can be used directly in denss.py and the Dmax will then be read from the header. The P(r) function will also be saved to a *_pr.dat file.
Using GNOM from ATSAS
One of the most popular software routines for performing this calculation is called GNOM, from the ATSAS suite of SAXS programs.
DENSS can natively accept GNOM output files (with a .out extension) as input to the denss.py script. An example of such a file, 6lyz.out, is provided along with the DENSS download. This file is a GNOM formatted file calculated from simulated SAXS data from PDB entry 6LYZ (hen egg white lysozyme). The small, 14 kDa globular particle provides for convenient and quick testing.
However, any valid procedure you like may be used for calculating the smooth fit to the scattering data. These smoothed curves can be given to DENSS as simple ASCII text files with three columns: q (=4π sin(θ)/λ in Å-1); intensity; and error on the intensity. DENSS will interpolate the given scattering profile to the necessary q values used in the reconstruction.
Note
It is not recommended to use the automated version of GNOM, called datgnom (or autognom). This version truncates the data to low resolution (something like q < 7/Rg) due to limitations of bead modeling algorithms (such as DAMMIF) that cannot utilize such higher resolution data. However DENSS is fully capable of using high resolution data (though due to information content may not improve the resolution of the final map). It is highly recommended to manually run GNOM (for example from within the Primus interface) and ensure that all high resolution data points are used. Even in cases where the high resolution data may not be the greatest quality, it is likely to be better than the simple Porod extrapolation that DENSS performs when the voxel sizes are smaller than the corresponding resolution of the data defined by qmax.