Tokenize V1.0.1

Changed

• Documentation updated for v1.0.0 options #46

Release 1.0.0

1.0.0

Added

• Support for numbers by @Annipoo #34
• Support for strings #37
• Code of Conduct #23

Changed

• Spaces are now tagged with space instead of spc #41
• Tokenization of numbers and strings is enabled by default #40
• Options are now processed by a more conventional means #39
• The location for the pack's home is updated

Release v0.1.2

Added PlDoc compliant documentation and set up pack.pl to facilitate the SWI-Prolog pack utility auto-detecting upgrades.

Minor correction to documentation and meta data.

v0.1.1

Corrected download url and version info in pack.pl

Initial release of the simple tokenizer

The first release of a simple tokenizer written in SWI-Prolog.