File: //usr/lib/python3/dist-packages/pygments/lexers/__pycache__/special.cpython-38.pyc
U
`a�[O � @ sz d Z ddlZddlmZ ddlmZmZmZ ddlm Z m
Z
m Z ddgZ G dd� de�Z
i Ze�d �ZG d
d� de�ZdS ) z�
pygments.lexers.special
~~~~~~~~~~~~~~~~~~~~~~~
Special lexers.
:copyright: Copyright 2006-2017 by the Pygments team, see AUTHORS.
:license: BSD, see LICENSE for details.
� N)�Lexer)�Token�Error�Text)�get_choice_opt� text_type�BytesIO� TextLexer�
RawTokenLexerc @ s: e Zd ZdZdZdgZdgZdgZdZdd� Z d d
� Z
d S ) r z3
"Null" lexer, doesn't highlight anything.
z Text only�textz*.txtz
text/plaing{�G�z�?c c s dt |fV d S )Nr )r )�selfr � r
�9/usr/lib/python3/dist-packages/pygments/lexers/special.py�get_tokens_unprocessed s z TextLexer.get_tokens_unprocessedc C s t jS )N)r �priority)r r
r
r � analyse_text# s zTextLexer.analyse_textN) �__name__�
__module__� __qualname__�__doc__�name�aliases� filenames� mimetypesr r r r
r
r
r r s s .*?
c @ s< e Zd ZdZdZdgZg ZdgZdd� Zdd� Z d d
� Z
d S ) r
aq
Recreate a token stream formatted with the `RawTokenFormatter`. This
lexer raises exceptions during parsing if the token stream in the
file is malformed.
Additional options accepted:
`compress`
If set to ``"gz"`` or ``"bz2"``, decompress the token stream with
the given compression algorithm before lexing (default: ``""``).
zRaw token data�rawz application/x-pygments-tokensc K s* t |dddddgd�| _tj| f|� d S )N�compress� Znone�gz�bz2)r r r �__init__)r Zoptionsr
r
r r < s
�zRawTokenLexer.__init__c c s� t |t�r|�d�}| jdkrDdd l}|�dddt|��}|�� }n | jdkr`dd l}|� |�}|�
d�d }| � |�D ]\}}}||fV qxd S ) N�asciir r r �rb� r �
) �
isinstancer �encoder �gzipZGzipFiler �readr �
decompress�stripr )r r r& Zgzipfiler �i�t�vr
r
r �
get_tokensA s
zRawTokenLexer.get_tokensc c s� d}t �|�D ]�}z|�� �dd�\}}W n( tk
rR |�� �dd�}t}Y nlX t�|�}|s�t }|�d�dd � }|D ]&}|r�|d �
� s�td��t ||�}q||t|< |dd � �d
�}|||fV |t |�7 }qd S ) Nr � � r �replace�.zmalformed token name� ���zunicode-escape)
�line_re�finditer�group�split�
ValueError�decoder � _ttype_cache�getr �isupper�getattr�len) r r Zlength�matchZttypestr�valZttypeZttypesZttype_r
r
r r S s&
z$RawTokenLexer.get_tokens_unprocessedN) r r r r r r r r r r- r r
r
r
r r
+ s )r �reZpygments.lexerr Zpygments.tokenr r r Z
pygments.utilr r r �__all__r r: �compiler4 r
r
r
r
r �<module> s