Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
/.github export-ignore
/tests export-ignore
/.gitattributes export-ignore
/.gitignore export-ignore
/phpunit.xml.dist export-ignore
51 changes: 51 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: Tests

on:
push:
branches: [master, dev]
pull_request:
workflow_dispatch:

permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
tests:
name: PHP ${{ matrix.php }} (${{ matrix.dependencies }})
runs-on: ubuntu-latest
continue-on-error: ${{ matrix.experimental }}
strategy:
fail-fast: false
matrix:
php: ['8.2', '8.3', '8.4', '8.5']
dependencies: [highest]
experimental: [false]
include:
- php: '8.2'
dependencies: lowest
experimental: false
- php: '8.6'
dependencies: highest
experimental: true
steps:
- uses: actions/checkout@v7

- uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php }}
extensions: mbstring
coverage: none

- name: Validate composer.json
run: composer validate --strict

- uses: ramsey/composer-install@v4
with:
dependency-versions: ${{ matrix.dependencies }}

- name: Run tests
run: vendor/bin/phpunit
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
vendor/
.vscode/
composer.lock
composer.lock
.phpunit.cache/
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2025 Edgaras

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
72 changes: 43 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,30 @@
# StrSim v1.1.1
# StrSim v1.2.0

A collection of string similarity and distance algorithms implemented in PHP with full Unicode and multibyte character support. This library provides standalone static methods for computing various similarity metrics, useful in natural language processing, fuzzy matching, spell checking, and bioinformatics.

## What's New in v1.1.1
## What's New in v1.2.0

### 🔧 **Fixed Naming Issues**
- **Fixed `Jaro::distance()`** - Previously returned similarity values (1.0 = identical), now correctly returns distance values (0.0 = identical)
- **Fixed `JaroWinkler::distance()`** - Previously returned similarity values (1.0 = identical), now correctly returns distance values (0.0 = identical)
No changes are needed in calling code. Method names, signatures, return types and exception messages are unchanged.

### ✨ **New Functions Added**
- **`Jaro::similarity()`** - Returns proper similarity values (1.0 = identical, 0.0 = completely different)
- **`JaroWinkler::similarity()`** - Returns proper similarity values (1.0 = identical, 0.0 = completely different)
### Compatibility
- PHP 8.2 is supported. The requirement was lowered from 8.3 because the code uses no PHP 8.3 feature. Tested on PHP 8.2, 8.3, 8.4 and 8.5.
- `ext-mbstring` is now listed in `composer.json`. Every method already needed it.

### 📚 **Improvements**
- **Better MongeElkan** - Fixed edge cases for empty string comparisons
### Fixes
- `Hamming::distance()` now rejects invalid UTF-8 with `InvalidArgumentException`, like the other classes. Before, it returned a number that differed between PHP 8.2 and 8.3+.
- `Cosine::similarity()` returns exactly `1.0` for identical strings and never returns more than `1.0`. Before, rounding could give `1.0000000000000002`. Other results change by at most about 3e-16.
- `Cosine::similarityFromVectors()` keeps results between `-1.0` and `1.0`. Arrays with different keys (for example after `array_filter()`) are compared by position instead of raising "Undefined array key" warnings. Arrays with the same keys are still matched by key. A `NAN` value no longer causes a warning on PHP 8.5.

### 🔄 **Migration Guide**
If you were using `Jaro::distance()` or `JaroWinkler::distance()` expecting similarity values (where 1.0 = identical):
- **Before**: `Jaro::distance("hello", "hello")` returned `1.0`
- **After**: Use `Jaro::similarity("hello", "hello")` to get `1.0`, or `Jaro::distance("hello", "hello")` returns `0.0`
### Performance
- Each string is split into characters once per call instead of calling `mb_substr()` for every character. The dynamic-programming algorithms keep two or three table rows instead of the full table. This change does not alter any result.
- One measurement on PHP 8.3 with two 2000-character strings: `Levenshtein::distance()` took 8949.9 ms and 70.5 MB in v1.1.1, and 163.4 ms and 0.3 MB in v1.2.0. Two 3000-character strings used to stop with a "memory exhausted" fatal error under the default 128 MB limit. They now return a result.

---

## Requirements

- PHP 8.3+
- PHP 8.2+
- mbstring extension
- Composer

## Installation
Expand All @@ -43,10 +43,10 @@ require __DIR__ . '/vendor/autoload.php';

## Features

- **Full Unicode Support**: All algorithms handle multibyte characters, emoji, combining marks, and complex grapheme clusters
- **UTF-8 Validation**: Automatic validation of input strings with clear error messages
- **Full Unicode Support**: All algorithms accept any valid UTF-8 text, including multibyte characters, emoji and combining marks
- **UTF-8 Validation**: Every string method throws `InvalidArgumentException` for invalid UTF-8
- **Error Handling**: Proper exception types with descriptive messages
- **Code-Point Based**: Consistent behavior across all Unicode normalization forms
- **Code-Point Based**: Strings are compared by Unicode code point without normalization
- **Optimized Tokenization**: Smart whitespace handling for text-based algorithms
- **Distance vs Similarity**: Clear distinction between distance measures (0 = identical) and similarity measures (1 = identical)

Expand All @@ -55,19 +55,19 @@ require __DIR__ . '/vendor/autoload.php';
| Class | Method | Return Range | Description |
|--------------------|----------------------------|--------------|----------------------------------------------------------------------|
| `Levenshtein` | `distance()` | 0 to ∞ | Number of insertions, deletions, or substitutions needed. |
| `DamerauLevenshtein` | `distance()` | 0 to ∞ | Levenshtein with transpositions included. |
| `DamerauLevenshtein` | `distance()` | 0 to ∞ | Optimal string alignment: Levenshtein plus adjacent transpositions. |
| `Hamming` | `distance()` | 0 to ∞ | Number of differing positions (requires equal-length strings). |
| `Jaro` | `similarity()` | 0.0 to 1.0 | Similarity based on character matches and transpositions. |
| `Jaro` | `distance()` | 0.0 to 1.0 | Distance measure (1 - similarity). |
| `JaroWinkler` | `similarity()` | 0.0 to 1.0 | Jaro with a prefix match boost for similar string starts. |
| `JaroWinkler` | `similarity()` | 0.0 to 1.0 | Jaro with a prefix match boost (`prefixScale` 0.0 to 0.25). |
| `JaroWinkler` | `distance()` | 0.0 to 1.0 | Distance measure (1 - similarity). |
| `LCS` | `length()` | 0 to ∞ | Length of the longest common subsequence. |
| `SmithWaterman` | `score()` | 0 to ∞ | Local alignment scoring for best-matching subsequences. |
| `NeedlemanWunsch` | `score()` | -∞ to ∞ | Global alignment scoring for entire string similarity. |
| `Cosine` | `similarity()` | 0.0 to 1.0 | Similarity via character frequency vectors. |
| `Cosine` | `similarityFromVectors()` | -1.0 to 1.0 | Cosine similarity for numeric vector inputs. |
| `Jaccard` | `index()` | 0.0 to 1.0 | Ratio of shared to total unique characters. |
| `MongeElkan` | `similarity()` | 0.0 to 1.0 | Average best-word similarity using Jaro-Winkler internally. |
| `MongeElkan` | `similarity()` | 0.0 to 1.0 | Average best-word Jaro-Winkler similarity of the first string's words. |

## Understanding Distance vs Similarity

Expand Down Expand Up @@ -119,25 +119,25 @@ JaroWinkler::similarity("martha", "marhta"); // Returns: 0.961 (similarity)
JaroWinkler::distance("martha", "marhta"); // Returns: 0.039 (distance = 1 - similarity)

// Finding common subsequence in DNA fragments
LCS::length("ACCGGTCGAGTGCGCGGAAGCCGGCCGAA", "GTCGTTCGGAATGCCGTTGCTCTGTAAA"); // Returns: 13
LCS::length("ACCGGTCGAGTGCGCGGAAGCCGGCCGAA", "GTCGTTCGGAATGCCGTTGCTCTGTAAA"); // Returns: 20

// Local alignment score for substring match
SmithWaterman::score("ACACACTA", "AGCACACA"); // Returns: 11
SmithWaterman::score("ACACACTA", "AGCACACA"); // Returns: 12

// Global alignment score for complete sequence match
NeedlemanWunsch::score("GATTACA", "GCATGCU"); // Returns: 0

// Comparing word frequency in short texts
// Comparing character frequency in short texts
Cosine::similarity("night", "nacht"); // Returns: 0.6

// Comparing embedding vectors from NLP model
Cosine::similarityFromVectors([0.1, 0.2, 0.3], [0.1, 0.3, 0.4]); // Returns: 0.925
Cosine::similarityFromVectors([0.1, 0.2, 0.3], [0.1, 0.3, 0.4]); // Returns: 0.996

// Comparing token overlap in short strings
// Comparing character-set overlap in short strings
Jaccard::index("abc", "bcd"); // Returns: 0.5

// Fuzzy match between two multi-word names
MongeElkan::similarity("john smith", "jon smythe"); // Returns: 0.822
MongeElkan::similarity("john smith", "jon smythe"); // Returns: 0.896
```

### Unicode and Multibyte Examples
Expand All @@ -154,8 +154,8 @@ Hamming::distance("👍🏽", "👍🏾"); // Returns: 1
// Different scripts and languages
Jaro::similarity("привет", "привет"); // Returns: 1.0 (identical)
Jaro::distance("привет", "привет"); // Returns: 0.0 (no distance)
JaroWinkler::similarity("عربي", "عربى"); // Returns: 0.9 (high similarity)
JaroWinkler::distance("عربي", "عربى"); // Returns: 0.1 (low distance)
JaroWinkler::similarity("عربي", "عربى"); // Returns: 0.883 (high similarity)
JaroWinkler::distance("عربي", "عربى"); // Returns: 0.117 (low distance)

// ZWJ sequences and combining marks
Levenshtein::distance("👨‍👩‍👧‍👦", "👨👩👧👦"); // Returns: 3
Expand Down Expand Up @@ -201,6 +201,20 @@ try {
}
```

## Notes

- Strings are compared by code point. No Unicode normalization is applied, so NFC and NFD forms of the same text differ.
- `DamerauLevenshtein` is the optimal string alignment variant (restricted edit distance).
- `MongeElkan::similarity($a, $b)` is not symmetric. It averages over the words of `$a`.
- `JaroWinkler` `prefixScale` must be between 0 and 0.25. Larger values return results outside 0 to 1.

## Testing

```bash
composer install
composer test
```

## Useful links

- [Levenshtein](https://en.wikipedia.org/wiki/Levenshtein_distance)
Expand Down
12 changes: 10 additions & 2 deletions composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,17 @@
},
"minimum-stability": "stable",
"require": {
"php": ">=8.3.0"
"php": ">=8.2",
"ext-mbstring": "*"
},
"require-dev": {
"phpunit/phpunit": "^11.5"
"phpunit/phpunit": "^11.5.50"
},
"scripts": {
"test": "phpunit"
},
"support": {
"issues": "https://github.com/Edgaras0x4E/StrSim/issues",
"source": "https://github.com/Edgaras0x4E/StrSim"
}
}
22 changes: 22 additions & 0 deletions phpunit.xml.dist
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
bootstrap="vendor/autoload.php"
cacheDirectory=".phpunit.cache"
colors="true"
failOnWarning="true"
failOnNotice="true"
failOnDeprecation="true"
failOnRisky="true"
beStrictAboutOutputDuringTests="true">
<testsuites>
<testsuite name="StrSim">
<directory>tests</directory>
</testsuite>
</testsuites>
<source>
<include>
<directory>src</directory>
</include>
</source>
</phpunit>
26 changes: 14 additions & 12 deletions src/Cosine.php
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ public static function similarity(string $a, string $b): float {
if (!mb_check_encoding($a, 'UTF-8') || !mb_check_encoding($b, 'UTF-8')) {
throw new \InvalidArgumentException("Input strings must be valid UTF-8.");
}

$tokensA = self::countMbChars($a);
$tokensB = self::countMbChars($b);
$dot = 0;
Expand All @@ -22,26 +22,23 @@ public static function similarity(string $a, string $b): float {
$normB += $v * $v;
}

return ($normA && $normB) ? $dot / (sqrt($normA) * sqrt($normB)) : 0;
return ($normA && $normB) ? min(1.0, $dot / sqrt($normA * $normB)) : 0.0;
}

private static function countMbChars(string $str): array {
$chars = [];
$length = mb_strlen($str, 'UTF-8');

for ($i = 0; $i < $length; $i++) {
$char = mb_substr($str, $i, 1, 'UTF-8');
$chars[$char] = ($chars[$char] ?? 0) + 1;
}

return $chars;
return array_count_values(mb_str_split($str, 1, 'UTF-8'));
}

public static function similarityFromVectors(array $vecA, array $vecB): float {
if (count($vecA) !== count($vecB)) {
throw new \InvalidArgumentException("Vectors must be the same length.");
}

if (array_diff_key($vecA, $vecB)) {
$vecA = array_values($vecA);
$vecB = array_values($vecB);
}

$dot = 0.0;
$normA = 0.0;
$normB = 0.0;
Expand All @@ -53,6 +50,11 @@ public static function similarityFromVectors(array $vecA, array $vecB): float {
$normB += $valB * $valB;
}

return ($normA && $normB) ? $dot / (sqrt($normA) * sqrt($normB)) : 0.0;
if ($normA != 0.0 && $normB != 0.0) {
$similarity = $dot / (sqrt($normA) * sqrt($normB));
return is_nan($similarity) ? $similarity : max(-1.0, min(1.0, $similarity));
}

return 0.0;
}
}
42 changes: 21 additions & 21 deletions src/DamerauLevenshtein.php
Original file line number Diff line number Diff line change
Expand Up @@ -6,36 +6,36 @@ public static function distance(string $a, string $b): int {
if (!mb_check_encoding($a, 'UTF-8') || !mb_check_encoding($b, 'UTF-8')) {
throw new \InvalidArgumentException("Input strings must be valid UTF-8.");
}

$lenA = mb_strlen($a, 'UTF-8');
$lenB = mb_strlen($b, 'UTF-8');
$dp = [];

for ($i = 0; $i <= $lenA; $i++) $dp[$i][0] = $i;
for ($j = 0; $j <= $lenB; $j++) $dp[0][$j] = $j;
$charsA = mb_str_split($a, 1, 'UTF-8');
$charsB = mb_str_split($b, 1, 'UTF-8');
$lenA = count($charsA);
$lenB = count($charsB);

$prev2 = [];
$prev = range(0, $lenB);

for ($i = 1; $i <= $lenA; $i++) {
$cur = [$i];
$charA = $charsA[$i - 1];
for ($j = 1; $j <= $lenB; $j++) {
$charA = mb_substr($a, $i - 1, 1, 'UTF-8');
$charB = mb_substr($b, $j - 1, 1, 'UTF-8');
$charB = $charsB[$j - 1];
$cost = ($charA === $charB) ? 0 : 1;
$dp[$i][$j] = min(
$dp[$i - 1][$j] + 1,
$dp[$i][$j - 1] + 1,
$dp[$i - 1][$j - 1] + $cost

$cur[$j] = min(
$prev[$j] + 1,
$cur[$j - 1] + 1,
$prev[$j - 1] + $cost
);

if ($i > 1 && $j > 1) {
$charA2 = mb_substr($a, $i - 2, 1, 'UTF-8');
$charB2 = mb_substr($b, $j - 2, 1, 'UTF-8');
if ($charA === $charB2 && $charA2 === $charB) {
$dp[$i][$j] = min($dp[$i][$j], $dp[$i - 2][$j - 2] + $cost);
}

if ($i > 1 && $j > 1 && $charA === $charsB[$j - 2] && $charsA[$i - 2] === $charB) {
$cur[$j] = min($cur[$j], $prev2[$j - 2] + $cost);
}
}
$prev2 = $prev;
$prev = $cur;
}

return $dp[$lenA][$lenB];
return $prev[$lenB];
}
}
Loading
Loading