Removing Tashkel

بسم الله الرحمن الرحيم

Introduction

The modern way for representing text characters is unicode (contrary to the old charsets), most modern programming languages and database systems support Unicode natively now.

Sticking to the standard is the best practice to maintain portability and continuous improvement. So using the Unicode category is better than using manual functions.

Each unicode character is assigned a category. The arabic tashkeel characters are defined in the ‘Mark, Nonspacing’ category (Mn).

Update:

A better way would be to leave only the letters in the {Lo} category which contains the basic arabic language.

Also leave spaces, tabs and newlines:

[^\p{Lo}\p{Z}\n]

Arabic characters in the Mark, Nonspacing

Unicode	Name	Character	Preview
U+0610	ARABIC SIGN SALLALLAHOU ALAYHE WASSALLAM	ؐ	view
U+0611	ARABIC SIGN ALAYHE ASSALLAM	ؑ	view
U+0612	ARABIC SIGN RAHMATULLAH ALAYHE	ؒ	view
U+0613	ARABIC SIGN RADI ALLAHOU ANHU	ؓ	view
U+0614	ARABIC SIGN TAKHALLUS	ؔ	view
U+0615	ARABIC SMALL HIGH TAH	ؕ	view
U+0616	ARABIC SMALL HIGH LIGATURE ALEF WITH LAM WITH YEH	ؖ	view
U+0617	ARABIC SMALL HIGH ZAIN	ؗ	view
U+0618	ARABIC SMALL FATHA	ؘ	view
U+0619	ARABIC SMALL DAMMA	ؙ	view
U+061A	ARABIC SMALL KASRA	ؚ	view
U+064B	ARABIC FATHATAN	ً	view
U+064C	ARABIC DAMMATAN	ٌ	view
U+064D	ARABIC KASRATAN	ٍ	view
U+064E	ARABIC FATHA	َ	view
U+064F	ARABIC DAMMA	ُ	view
U+0650	ARABIC KASRA	ِ	view
U+0651	ARABIC SHADDA	ّ	view
U+0652	ARABIC SUKUN	ْ	view
U+0653	ARABIC MADDAH ABOVE	ٓ	view
U+0654	ARABIC HAMZA ABOVE	ٔ	view
U+0655	ARABIC HAMZA BELOW	ٕ	view
U+0656	ARABIC SUBSCRIPT ALEF	ٖ	view
U+0657	ARABIC INVERTED DAMMA	ٗ	view
U+0658	ARABIC MARK NOON GHUNNA	٘	view
U+0659	ARABIC ZWARAKAY	ٙ	view
U+065A	ARABIC VOWEL SIGN SMALL V ABOVE	ٚ	view
U+065B	ARABIC VOWEL SIGN INVERTED SMALL V ABOVE	ٛ	view
U+065C	ARABIC VOWEL SIGN DOT BELOW	ٜ	view
U+065D	ARABIC REVERSED DAMMA	ٝ	view
U+065E	ARABIC FATHA WITH TWO DOTS	ٞ	view
U+065F	ARABIC WAVY HAMZA BELOW	ٟ	view
U+0670	ARABIC LETTER SUPERSCRIPT ALEF	ٰ	view
U+06D6	ARABIC SMALL HIGH LIGATURE SAD WITH LAM WITH ALEF MAKSURA	ۖ	view