Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,52 @@ if !result.all_correct() && result.repair_possible {
}
```

### In-memory repair (no file I/O)

`Repair` rebuilds missing slices from slices you already hold in memory, such
as article bodies from a Usenet download, plus recovery blocks. It reads and
writes no files. Feed each present slice once, in any order, then solve:

```rust
use rust_par2::recovery::RecoveryBlock;
use rust_par2::{BlockMap, Repair};
# fn demo(file_set: &rust_par2::Par2FileSet, recovery: Vec<RecoveryBlock>,
# present: Vec<(usize, Vec<u8>)>, mut missing: Vec<usize>) -> Result<(), rust_par2::RepairError> {
// Global slice indices run across files in the Main packet's file order.
let map = BlockMap::new(file_set);
let slice_size = file_set.slice_size as usize;

let mut repair = Repair::new_owning(recovery, slice_size, map.total_blocks as usize)?;

for (index, bytes) in &present {
// Verify before feeding: a corrupt slice cannot be taken back out.
let loc = map.global_to_file(*index).unwrap();
let file = &file_set.files[&loc.file.file_id];
if file.slice_matches(loc.local_index as usize, bytes, file_set.slice_size) {
repair.feed(*index, bytes)?; // the last slice of a file may be short
} else {
missing.push(*index);
}
}
repair.truncate_rows(missing.len()); // optional: free rows solve won't use

// Every slice must be either fed or listed as missing.
for (index, slice) in repair.solve(&missing)? {
let loc = map.global_to_file(index).unwrap();
// `slice` is zero-padded to slice_size; write only loc.len bytes at loc.offset.
let _ = (&loc.file.filename, loc.offset, &slice[..loc.len]);
}
# Ok(()) }
```

`solve` fails with `RepairError::InsufficientRecovery` when more slices are
missing than recovery rows are loaded. Peak memory is `rows 脳 slice_size` for
the accumulators plus `missing 脳 slice_size` for the output, plus whatever you
buffer before feeding. `feed_batch` takes several slices at once and uses the
same cache-tiled SIMD path as the file-based repair, which is built on `Repair`.
`SliceHasher` computes the IFSC MD5 + CRC32 pair incrementally if a slice
arrives in pieces.

## Performance

Benchmarked against [par2cmdline-turbo](https://github.com/animetosho/par2cmdline-turbo)
Expand Down Expand Up @@ -221,6 +267,9 @@ than treating the entire file as lost.
| `repair(file_set, dir)` | Verify + repair in one call |
| `repair_from_verify(file_set, dir, verify_result)` | Repair with pre-computed verify |
| `compute_hash_16k(path)` | MD5 of first 16 KiB (for file identification) |
| `Repair::new_owning(recovery, slice_size, total)` | In-memory repair builder: `feed`, `feed_batch`, `solve` |
| `BlockMap::new(file_set)` | Map global slice indices to (file, offset, length) |
| `SliceHasher::checksum(data, slice_size)` | IFSC MD5 + CRC32 of a slice (zero-padded) |

### Key types

Expand All @@ -243,6 +292,8 @@ than treating the entire file as lost.
| `RepairError::SingularMatrix` | Recovery blocks are linearly dependent |
| `RepairError::NoDamage` | Nothing to repair (all files intact) |
| `RepairError::VerifyFailed` | Post-repair verification failed |
| `RepairError::SliceOutOfRange` / `SliceTooLong` / `DuplicateSlice` / `IncompleteFeed` | Invalid input to the `Repair` builder |
| `RepairError::InvalidSliceSize` / `InvalidRecoveryBlock` | Bad `Repair` construction arguments |

## Dependencies

Expand Down
5 changes: 4 additions & 1 deletion src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,16 @@ pub mod matrix;
mod packets;
pub mod recovery;
pub mod repair;
pub mod slice;
pub mod types;
mod verify;

pub use packets::{ParseError, parse_par2_file as parse, parse_par2_reader};
pub use repair::{
RepairError, RepairResult, repair, repair_from_verify, repair_from_verify_no_reverify,
BlockFile, BlockMap, Repair, RepairError, RepairResult, SliceLocation, build_block_map, repair,
repair_from_verify, repair_from_verify_no_reverify,
};
pub use slice::SliceHasher;
pub use types::{
DamagedFile, Md5Hash, MissingFile, Par2File, Par2FileSet, SliceChecksum, VerifiedFile,
VerifyResult,
Expand Down
2 changes: 1 addition & 1 deletion src/recovery.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ use crate::packets::{HEADER_SIZE, MAGIC};
const TYPE_RECOVERY: &[u8; 16] = b"PAR 2.0\x00RecvSlic";

/// A recovery block extracted from a RecoverySlice packet.
#[derive(Debug)]
#[derive(Debug, Clone)]
pub struct RecoveryBlock {
/// The exponent (recovery block index) used in the Vandermonde matrix.
pub exponent: u32,
Expand Down
Loading