mirror of
https://github.com/react/react-native-devtools-frontend.git
synced 2026-09-30 01:07:01 +08:00
Tim van der Lippe
e4974ddc4c
json.dump appears to add extraneous spaces before newline characters. By specifying the separators option, we can remove these whitespaces, reducing the diff noise. Solution taken from https://stackoverflow.com/questions/16311562/python-json-without-whitespaces DISABLE_THIRD_PARTY_CHECK=Fix whitespace in node_modules Change-Id: I26556e5053f19a9021ad41779319bdfe3f8c2b6c Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2013154 Commit-Queue: Tim van der Lippe <tvanderlippe@chromium.org> Commit-Queue: Mathias Bynens <mathias@chromium.org> Auto-Submit: Tim van der Lippe <tvanderlippe@chromium.org> Reviewed-by: Mathias Bynens <mathias@chromium.org>
chardet 
Chardet is a character detection module for NodeJS written in pure Javascript. Module is based on ICU project http://site.icu-project.org/, which uses character occurency analysis to determine the most probable encoding.
Installation
npm i chardet
Usage
To return the encoding with the highest confidence:
var chardet = require('chardet');
chardet.detect(Buffer.alloc('hello there!'));
// or
chardet.detectFile('/path/to/file', function(err, encoding) {});
// or
chardet.detectFileSync('/path/to/file');
To return the full list of possible encodings:
var chardet = require('chardet');
chardet.detectAll(Buffer.alloc('hello there!'));
// or
chardet.detectFileAll('/path/to/file', function(err, encoding) {});
// or
chardet.detectFileAllSync('/path/to/file');
//Returned value is an array of objects sorted by confidence value in decending order
//e.g. [{ confidence: 90, name: 'UTF-8'}, {confidence: 20, name: 'windows-1252', lang: 'fr'}]
Working with large data sets
Sometimes, when data set is huge and you want to optimize performace (in tradeoff of less accuracy), you can sample only first N bytes of the buffer:
chardet.detectFile('/path/to/file', { sampleSize: 32 }, function(err, encoding) {});
Supported Encodings:
- UTF-8
- UTF-16 LE
- UTF-16 BE
- UTF-32 LE
- UTF-32 BE
- ISO-2022-JP
- ISO-2022-KR
- ISO-2022-CN
- Shift-JIS
- Big5
- EUC-JP
- EUC-KR
- GB18030
- ISO-8859-1
- ISO-8859-2
- ISO-8859-5
- ISO-8859-6
- ISO-8859-7
- ISO-8859-8
- ISO-8859-9
- windows-1250
- windows-1251
- windows-1252
- windows-1253
- windows-1254
- windows-1255
- windows-1256
- KOI8-R
Currently only these encodings are supported, more will be added soon.