DENSS v 1.6.3 released today has several bug fixes and feature updates:
Major/moderate updates:
- Updated output fit files from denss.fit_data.py to have the .fit extension common for such file types. This replaces the old *_fit.dat suffix.
- .fit files created by denss.fit_data.py and denss.py now conform to the standard .fit format as a 4-column ascii text file containing (q, I, error, fit).
- When denss.py is given a file with real data and a fit, such as a GNOM .out file or the .fit file mentioned above, the output fit.png plot will show both the raw data and error bars and the fitted profile, superimposed with the interpolated data points used for the reconstruction and the corresponding fit from the density.
- denss.py and denss.all.py logging was updated to include more information in all log files, including a snapshot of the standard output for every 500 steps. Various stages of the denss.all.py pipeline (e.g. when running reconstructions, alignment, averaging, etc.) are also now saved in the log file to note when they occur with timestamps.
- Added some dedicated scripts for performing individual stages of denss.all.py including denss.select_enantiomer.py (for selecting the best enantiomer from a single map), denss.select_enantiomers.py (for selecting the best enantiomers from a set of multiple maps), and denss.generate_reference.py (for generating a reference from a set of maps using the binary averaging procedure). Having these dedicated scripts separate from denss.all.py is helpful for users who want more control over these steps, in particular for users who would like to utilize multi-node clusters for faster computations (the built-in python multiprocessing module doesn’t work well over separate nodes, only over multiple cores on the same CPU).
- Added resolution option to denss.pdb2mrc.py, which is implemented as a B-factor in the real space form factor equation (which uses the Cromer-Mann 5-term Gaussian summation). This is much more accurate than thee simple Gaussian sphere approximation that is standard for many other calculations of density from atomic coordinates and results in a relatively accurate scattering profile (though still ignoring some very non-trivial solvent effects).
- Added –add_noise option to denss.refine.py. This allows one to add noise to the starting map, uniformly distributed throughout the box volume. The scale of the noise is given as a scalar value after the –add_noise option.
Minor updates and bug fixes:
- denss.py now saves the .fit file when given raw data (the automated fitting of raw data was a feature added in v 1.6.1, this update just saves the .fit file which wasn’t previously done).
- The _fit.png plot was updated to remove the Porod extrapolation to the data at high-q values not included in the data, as this was often obscuring low-q values in the plot, which are more important for assessing the quality of the low-resolution reconstructions.
- Resolution estimate now saved in header of output fsc file for several scripts (denss.all.py, denss.average.py, etc).
- Improved alignment a bit by removing interpolation from coarse alignment step when recentering, now done simply as a roll of the array rather than interpolation of the translation. The fine alignment step still must use interpolation so this effectively removes an unnecessary interpolation step.
- Improved enantiomer selection by removing some unnecessary alignment steps which add interpolation.
- Added option to denss.mrcops.py to generate an enantiomer with a simple z-axis flip (the –zflip option).
- Limited generate_reference() function to use at most eight maps rather than the maximum of 2^n maps. This makes the calculation much faster without impacting results significantly.
- Refactored parallelization when performing enantiomer selection for many maps (e.g. as in denss.all.py) to take better advantage of multi-core systems. Now parallelization occurs over N maps (which can theoretically use up to N cores) rather than parallelizing the enantiomer selection itself (which can only parallelize up to two cores).
- Added ability to at least occasionally calculate Rg when running on the GPU. Cannot currently calculate Rg on the GPU easily, only on the CPU, so this was previously never calculated (as this requires copying to the CPU, which is a significant bottleneck in speed). However, Rg is now updated whenever shrink-wrap is run, since shrink-wrap must be calculated on the CPU anyways.
- Fixed bugs with GPU implementation when loading CuPy.
- Fixed bug in reported correlation score for comparing maps.
- Fixed bug in symmetry averaging where the zeroth symmetry mate was mistakenly calculated and added twice, giving unequal weight when averaging.
- Other miscellaneous minor bug fixes.