1#@!#!123s

: / proc / self / root / usr / lib / python3.6 / site-packages / chardet / __pycache__ /

Filename : charsetprober.cpython-36.pyc

back

3

�]9Y�������������������@���s0���d�dl�Z�d�dlZddlmZ�G�dd��de�ZdS�)�����N����)�ProbingStatec���������������@���sn���e�Zd�ZdZddd�Zdd��Zedd���Zd	d
��Zedd���Z	d
d��Z
edd���Zedd���Z
edd���ZdS�)�
CharSetProbergffffff�?Nc�������������C���s���d�|�_�||�_tjt�|�_d�S�)N)�_state�lang_filter�loggingZ	getLogger�__name__Zlogger)�selfr�����r
����#/usr/lib/python3.6/charsetprober.py�__init__'���s����zCharSetProber.__init__c�������������C���s���t�j|�_d�S�)N)r���Z	DETECTINGr���)r	���r
���r
���r����reset,���s����zCharSetProber.resetc�������������C���s���d�S�)Nr
���)r	���r
���r
���r����charset_name/���s����zCharSetProber.charset_namec�������������C���s���d�S�)Nr
���)r	����bufr
���r
���r����feed3���s����zCharSetProber.feedc�������������C���s���|�j�S�)N)r���)r	���r
���r
���r����state6���s����zCharSetProber.statec�������������C���s���dS�)Ng��������r
���)r	���r
���r
���r����get_confidence:���s����zCharSetProber.get_confidencec�������������C���s���t�jdd|��}�|�S�)Ns���([�-])+���� )�re�sub)r���r
���r
���r����filter_high_byte_only=���s����z#CharSetProber.filter_high_byte_onlyc�������������C���sb���t���}tjd|��}xJ|D�]B}|j|dd����|dd��}|j���rP|dk�rPd}|j|��qW�|S�)u9��
        We define three types of bytes:
        alphabet: english alphabets [a-zA-Z]
        international: international characters [-ÿ]
        marker: everything else [^a-zA-Z-ÿ]

        The input buffer can be thought to contain a series of words delimited
        by markers. This function works to filter all words that contain at
        least one international character. All contiguous sequences of markers
        are replaced by a single space ascii character.

        This filter applies to all scripts which do not use English characters.
        s%���[a-zA-Z]*[�-�]+[a-zA-Z]*[^a-zA-Z�-�]?Nr��������r������r���)�	bytearrayr����findall�extend�isalpha)r����filteredZwordsZwordZ	last_charr
���r
���r����filter_international_wordsB���s����
z(CharSetProber.filter_international_wordsc�������������C���s����t���}d}d}x�tt|���D�]r}|�||d���}|dkr>d}n|dkrJd}|dk�r|j���r||kr�|�r�|j|�||����|jd��|d�}qW�|s�|j|�|d	����|S�)
a���
        Returns a copy of ``buf`` that retains only the sequences of English
        alphabet and high byte characters that are not between <> characters.
        Also retains English alphabet and high byte characters immediately
        before occurrences of >.

        This filter can be applied to all scripts which contain both English
        characters and extended ASCII characters, but is currently only used by
        ``Latin1Prober``.
        Fr���r�������>����<Tr���r���N)r����range�lenr���r���)r���r���Zin_tag�prevZcurrZbuf_charr
���r
���r����filter_with_english_lettersg���s"����
z)CharSetProber.filter_with_english_letters)N)r����
__module__�__qualname__ZSHORTCUT_THRESHOLDr���r
����propertyr���r���r���r����staticmethodr���r���r$���r
���r
���r
���r���r���#���s���
%r���)r���r���Zenumsr����objectr���r
���r
���r
���r����<module>���s���

Name	Size	Last Modified	Owner	Permissions
__init__.cpython-36.opt-1.pyc	0.747 KB	October 12 2019 7:18:01	root	0644
__init__.cpython-36.pyc	0.747 KB	October 12 2019 7:18:01	root	0644
big5freq.cpython-36.opt-1.pyc	53.371 KB	October 12 2019 7:18:01	root	0644
big5freq.cpython-36.pyc	53.371 KB	October 12 2019 7:18:01	root	0644
big5prober.cpython-36.opt-1.pyc	1.017 KB	October 12 2019 7:18:01	root	0644
big5prober.cpython-36.pyc	1.017 KB	October 12 2019 7:18:01	root	0644
chardistribution.cpython-36.opt-1.pyc	6.091 KB	October 12 2019 7:18:01	root	0644
chardistribution.cpython-36.pyc	6.091 KB	October 12 2019 7:18:01	root	0644
charsetgroupprober.cpython-36.opt-1.pyc	2.098 KB	October 12 2019 7:18:01	root	0644
charsetgroupprober.cpython-36.pyc	2.098 KB	October 12 2019 7:18:01	root	0644
charsetprober.cpython-36.opt-1.pyc	3.295 KB	October 12 2019 7:18:01	root	0644
charsetprober.cpython-36.pyc	3.295 KB	October 12 2019 7:18:01	root	0644
codingstatemachine.cpython-36.opt-1.pyc	2.739 KB	October 12 2019 7:18:01	root	0644
codingstatemachine.cpython-36.pyc	2.739 KB	October 12 2019 7:18:01	root	0644
compat.cpython-36.opt-1.pyc	0.274 KB	October 12 2019 7:18:01	root	0644
compat.cpython-36.pyc	0.274 KB	October 12 2019 7:18:01	root	0644
cp949prober.cpython-36.opt-1.pyc	1.023 KB	October 12 2019 7:18:01	root	0644
cp949prober.cpython-36.pyc	1.023 KB	October 12 2019 7:18:01	root	0644
enums.cpython-36.opt-1.pyc	2.479 KB	October 12 2019 7:18:01	root	0644
enums.cpython-36.pyc	2.479 KB	October 12 2019 7:18:01	root	0644
escprober.cpython-36.opt-1.pyc	2.471 KB	October 12 2019 7:18:01	root	0644
escprober.cpython-36.pyc	2.471 KB	October 12 2019 7:18:01	root	0644
escsm.cpython-36.opt-1.pyc	7.116 KB	October 12 2019 7:18:01	root	0644
escsm.cpython-36.pyc	7.116 KB	October 12 2019 7:18:01	root	0644
eucjpprober.cpython-36.opt-1.pyc	2.279 KB	October 12 2019 7:18:01	root	0644
eucjpprober.cpython-36.pyc	2.279 KB	October 12 2019 7:18:01	root	0644
euckrfreq.cpython-36.opt-1.pyc	23.475 KB	October 12 2019 7:18:01	root	0644
euckrfreq.cpython-36.pyc	23.475 KB	October 12 2019 7:18:01	root	0644
euckrprober.cpython-36.opt-1.pyc	1.024 KB	October 12 2019 7:18:01	root	0644
euckrprober.cpython-36.pyc	1.024 KB	October 12 2019 7:18:01	root	0644
euctwfreq.cpython-36.opt-1.pyc	53.38 KB	October 12 2019 7:18:01	root	0644
euctwfreq.cpython-36.pyc	53.38 KB	October 12 2019 7:18:01	root	0644
euctwprober.cpython-36.opt-1.pyc	1.024 KB	October 12 2019 7:18:01	root	0644
euctwprober.cpython-36.pyc	1.024 KB	October 12 2019 7:18:01	root	0644
gb2312freq.cpython-36.opt-1.pyc	37.405 KB	October 12 2019 7:18:01	root	0644
gb2312freq.cpython-36.pyc	37.405 KB	October 12 2019 7:18:01	root	0644
gb2312prober.cpython-36.opt-1.pyc	1.032 KB	October 12 2019 7:18:01	root	0644
gb2312prober.cpython-36.pyc	1.032 KB	October 12 2019 7:18:01	root	0644
hebrewprober.cpython-36.opt-1.pyc	2.823 KB	October 12 2019 7:18:01	root	0644
hebrewprober.cpython-36.pyc	2.823 KB	October 12 2019 7:18:01	root	0644
jisfreq.cpython-36.opt-1.pyc	43.405 KB	October 12 2019 7:18:01	root	0644
jisfreq.cpython-36.pyc	43.405 KB	October 12 2019 7:18:01	root	0644
jpcntx.cpython-36.opt-1.pyc	37.682 KB	October 12 2019 7:18:01	root	0644
jpcntx.cpython-36.pyc	37.682 KB	October 12 2019 7:18:01	root	0644
langbulgarianmodel.cpython-36.opt-1.pyc	24.22 KB	October 12 2019 7:18:01	root	0644
langbulgarianmodel.cpython-36.pyc	24.22 KB	October 12 2019 7:18:01	root	0644
langcyrillicmodel.cpython-36.opt-1.pyc	29.641 KB	October 12 2019 7:18:01	root	0644
langcyrillicmodel.cpython-36.pyc	29.641 KB	October 12 2019 7:18:01	root	0644
langgreekmodel.cpython-36.opt-1.pyc	23.905 KB	October 12 2019 7:18:01	root	0644
langgreekmodel.cpython-36.pyc	23.905 KB	October 12 2019 7:18:01	root	0644
langhebrewmodel.cpython-36.opt-1.pyc	22.786 KB	October 12 2019 7:18:01	root	0644
langhebrewmodel.cpython-36.pyc	22.786 KB	October 12 2019 7:18:01	root	0644
langhungarianmodel.cpython-36.opt-1.pyc	24.194 KB	October 12 2019 7:18:01	root	0644
langhungarianmodel.cpython-36.pyc	24.194 KB	October 12 2019 7:18:01	root	0644
langthaimodel.cpython-36.opt-1.pyc	22.766 KB	October 12 2019 7:18:01	root	0644
langthaimodel.cpython-36.pyc	22.766 KB	October 12 2019 7:18:01	root	0644
langturkishmodel.cpython-36.opt-1.pyc	22.783 KB	October 12 2019 7:18:01	root	0644
langturkishmodel.cpython-36.pyc	22.783 KB	October 12 2019 7:18:01	root	0644
latin1prober.cpython-36.opt-1.pyc	2.795 KB	October 12 2019 7:18:01	root	0644
latin1prober.cpython-36.pyc	2.795 KB	October 12 2019 7:18:01	root	0644
mbcharsetprober.cpython-36.opt-1.pyc	2.103 KB	October 12 2019 7:18:01	root	0644
mbcharsetprober.cpython-36.pyc	2.103 KB	October 12 2019 7:18:01	root	0644
mbcsgroupprober.cpython-36.opt-1.pyc	1.02 KB	October 12 2019 7:18:01	root	0644
mbcsgroupprober.cpython-36.pyc	1.02 KB	October 12 2019 7:18:01	root	0644
mbcssm.cpython-36.opt-1.pyc	17.087 KB	October 12 2019 7:18:01	root	0644
mbcssm.cpython-36.pyc	17.087 KB	October 12 2019 7:18:01	root	0644
sbcharsetprober.cpython-36.opt-1.pyc	2.838 KB	October 12 2019 7:18:01	root	0644
sbcharsetprober.cpython-36.pyc	2.838 KB	October 12 2019 7:18:01	root	0644
sbcsgroupprober.cpython-36.opt-1.pyc	1.498 KB	October 12 2019 7:18:01	root	0644
sbcsgroupprober.cpython-36.pyc	1.498 KB	October 12 2019 7:18:01	root	0644
sjisprober.cpython-36.opt-1.pyc	2.305 KB	October 12 2019 7:18:01	root	0644
sjisprober.cpython-36.pyc	2.305 KB	October 12 2019 7:18:01	root	0644
universaldetector.cpython-36.opt-1.pyc	5.62 KB	October 12 2019 7:18:01	root	0644
universaldetector.cpython-36.pyc	5.62 KB	October 12 2019 7:18:01	root	0644
utf8prober.cpython-36.opt-1.pyc	1.847 KB	October 12 2019 7:18:01	root	0644
utf8prober.cpython-36.pyc	1.847 KB	October 12 2019 7:18:01	root	0644
version.cpython-36.opt-1.pyc	0.352 KB	October 12 2019 7:18:01	root	0644
version.cpython-36.pyc	0.352 KB	October 12 2019 7:18:01	root	0644

2017 © D7net \| D704T team