## [v1.4.0](https://gitlab.com/bfr_bioinformatics/AQUAMIS/compare/v1.3.7...v1.4.0) (2023-09-14)

### Upgrade Steps
* replace an old conda env with a new one based on [aquamis.yaml](envs/aquamis.yaml); a simple `mamba update` will break the installation without [pinning](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-pkgs.html#preventing-packages-from-updating-pinning) confindr to v0.7.4
* create a new [Snakemake profile](profiles) or edit the default [bfr.hpc/config.yaml](profiles/bfr.hpc/config.yaml) to match the IT resources.
  Usage of a personalized profile may be hard-coded in the wrapper [aquamis.py](aquamis.py) (search for line: `smk_profile = os.path.join(repo_path, "profiles", "bfr.hpc")` and replace "bfr.hpc", accordingly)
* **[WARNING]** rerun of this _new_ AQUAMIS setup on _old_ run folders will recalculate and **overwrite** preexisting data  

### Breaking Changes
* change of result JSON structure
* change of run-specific config YAML structure
* filename change of result JSON, run config and pipeline status files to include workflow name
* modified QUAST v5.2 result parsing
* use of [Snakemake profiles](https://snakemake.readthedocs.io/en/stable/executing/cli.html#profiles) to adapt to computing environment
* update to Snakemake v7 changes [rerun behaviour](https://github.com/snakemake/snakemake/issues/1694); the old strategy (`--rerun-triggers mtime`) may be configured via profiles

### New Features
* optional use of containerized AQUAMIS via [get_aquamis_docker.sh](docker/get_aquamis_docker.sh) and `docker-compose`
* scripted execution on multiple runs with combo [aquamis_wrapper.sh](scripts/aquamis_wrapper.sh) and [aquamis_runs.tsv](scripts/aquamis_runs.tsv)
* easier installation with [aquamis_setup.sh](scripts/aquamis_setup.sh)
* new confindr option to restrict to rMLST-only DBs: `aquamis.py --confindr_use_rmlst`
* result aggregation in the JSON files logs module execution time and links results via fastx hashing to the same sample
* download cache for MASH based reference assemblies from NCBI; a complete cache allows offline usage; a cache archive (size: 12G) can be provided upon request
* `<run>/json/filtered` are a JSON schema filtrate of `<run>/json/post-qc` based on [AQUAMIS_schema_filter_v20230906.json](resources/AQUAMIS_schema_filter_v20230906.json) without large histogram table data
* catch pre-assembly fails
* new `aquamis.py` arguments:
  * `--log` to specify logging directory, i.e. the prefix for `/.snakemake/log/datetime.snakemake.log` files
  * `--batch` for chunked analysis of large sample sheets
  * `--ephemeral` to remove most intermediate files for a smaller result data footprint
  * `--json` for high-throughput execution; includes argument ephemeral and stops after JSON creation
  * `--single_end` for single-end data
  * `--iontorrent` for single-end Ion Torrent data
  * `--version` for hard-coded version and git version (for commit distance to last tag)

### Bug Fixes
* fix JSON parsing after QUAST update to 5.2
* each executing user has their own temp directory, e.g. for shovill, to avoid permission errors on multi-user systems
* various [issues](https://gitlab.com/bfr_bioinformatics/AQUAMIS/-/issues)

### Performance Improvements
* Snakemake profile [nris.saga.slurm.normal](profiles/nris.saga.slurm.normal) is an example for AQUAMIS execution on SLURM-managed HPCs with Snakemake as slurm job orchestrator (tested on a 12k+ sample set) 
* setup archives are now retrieved from Gitlab LFS 

### Major Changes
* updated [AQUAMIS thresholds](resources/AQUAMIS_thresholds.json) and [JSON schema (filter)](resources/AQUAMIS_schema_v20230906.json)
* the last base of each read is trimmed by default in the fastp module 
* module timestamps are now derived from smk input files

### Other Changes
* harmonized python/smk/bash codestyle across BfR pipelines
* stabilize key order in mlst module
* sort bracken tsv results descending
* better logging for Snakemake report rules, confindr
* added create_pass_samplesheet.sh
* updated public test dataset with six different genera of pathogens
* QUAST commandline options "min-contig" and "min-identity" exposed
* scan for downsampling in shovill log; added FLASH metrics to shovill section in JSON
* rounding of various values in the (tsv) reports
* added [kraken2_db_hashes.json](resources/kraken2_db_hashes.json) for easier DB identification