HOME

Mini Shell 1.0
DIR:/snap/core/current/usr/lib/python3.5/lib2to3/__pycache__/
Current File : //snap/core/current/usr/lib/python3.5/lib2to3/__pycache__/refactor.cpython-35.pyc


���f�m�@s�dZddlmZdZddlZddlZddlZddlZddlZddl	Z	ddl
mZddlm
Z
mZmZddlmZdd	lmZmZdd
lmZddlmZdd
d�ZGdd�de�Zdd�Zdd�Zdd�Zdd�Zej ddfkrpddl!Z!e!j"Z#dd�Z$dd�Z%ne"Z#eZ$eZ%dd�Z&Gd d!�d!e�Z'Gd"d#�d#e(�Z)Gd$d%�d%e�Z*Gd&d'�d'e)�Z+dS)(z�Refactoring framework.

Used as a main program, this can refactor any number of files and/or
recursively descend down directories.  Imported as a module, this
provides infrastructure to write your own refactoring tool.
�)�with_statementz#Guido van Rossum <guido@python.org>N)�chain�)�driver�tokenize�token)�	find_root)�pytree�pygram)�	btm_utils)�btm_matcherTcCs�t|ggdg�}tjj|j�}g}xhttj|��D]Q}|jd�rI|jd�rI|r�|dd�}|j	|dd��qIW|S)zEReturn a sorted list of all available fix names in the given package.�*�fix_z.py�N����)
�
__import__�os�path�dirname�__file__�sorted�listdir�
startswith�endswith�append)Z	fixer_pkgZ
remove_prefixZpkgZ	fixer_dirZ	fix_names�name�r�&/usr/lib/python3.5/lib2to3/refactor.py�get_all_fix_names!src@seZdZdS)�
_EveryNodeN)�__name__�
__module__�__qualname__rrrrr .sr cCs�t|tjtjf�r:|jdkr0t�|jhSt|tj�rh|jrbt|j�St�t|tj	�r�t
�}x5|jD]*}x!|D]}|jt|��q�Wq�W|Std|��dS)zf Accepts a pytree Pattern Node and returns a set
        of the pattern types which will match first. Nz$Oh no! I don't understand pattern %s)
�
isinstancer	ZNodePatternZLeafPattern�typer ZNegatedPatternZcontent�_get_head_typesZWildcardPattern�set�update�	Exception)Zpat�r�p�xrrrr&2s
	
	
r&cCstjt�}g}x�|D]�}|jr�yt|j�}Wntk
r_|j|�Yq�XxU|D]}||j|�qgWq|jdk	r�||jj|�q|j|�qWx:tt	j
jj�t	j
j
�D]}||j|�q�Wt|�S)z^ Accepts a list of fixers and returns a dictionary
        of head node type --> fixer list.  N)�collections�defaultdict�list�patternr&r rZ_accept_typerr
�python_grammarZ
symbol2number�values�tokens�extend�dict)Z
fixer_listZ
head_nodesZevery�fixerZheadsZ	node_typerrr�_get_headnode_dictNs"
	

r7cs �fdd�t�d�D�S)zN
    Return the fully qualified names for fixers in the package pkg_name.
    csg|]}�d|�qS)�.r)�.0�fix_name)�pkg_namerr�
<listcomp>ks	z+get_fixers_from_package.<locals>.<listcomp>F)r)r;r)r;r�get_fixers_from_packagegsr=cCs|S)Nr)�objrrr�	_identitynsr?rcCs|jdd�S)Nz
�
)�replace)�inputrrr�_from_system_newlinesusrCcCs*tjdkr"|jdtj�S|SdS)Nr@)r�lineseprA)rBrrr�_to_system_newlineswsrEcs�d}tjtj|�j���fdd�}ttjtjtj	h�}t
�}yVxO|�\}}||kr�q`q`|tjkr�|r�Pd}q`|tjkr�|dkr�|�\}}|tjks�|dkr�P|�\}}|tjks|dkrP|�\}}|tj
krJ|dkrJ|�\}}x^|tjkr�|j|�|�\}}|tj
ks�|d	kr�P|�\}}qMWq`Pq`WWntk
r�YnXt|�S)
NFcst��}|d|dfS)Nrr)�next)�tok)�genrr�advance�sz(_detect_future_features.<locals>.advanceT�from�
__future__�import�(�,)r�generate_tokens�io�StringIO�readline�	frozensetr�NEWLINE�NL�COMMENTr'�STRING�NAME�OP�add�
StopIteration)�sourceZhave_docstringrI�ignore�features�tp�valuer)rHr�_detect_future_features�sD		
	
rac@seZdZdZdS)�
FixerErrorzA fixer could not be loaded.N)r!r"r#�__doc__rrrrrb�srbc@sieZdZddddiZdZdZdddd�Zd	d
�Zdd�Zd
d�Z	dd�Z
dd�Zdddd�Zdddd�Z
dd�Zdddd�Zdd�Zddd�Zdd �Zd!d"�Zdddd#d$�Zdd%d&�Zd'Zd(Zd)d*�Zd+d,�Zd-d.�Zd/d0�Zd1d2�Zd3d4�ZdS)5�RefactoringTool�print_functionF�write_unchanged_filesZFixrNcCs�||_|pg|_|jj�|_|dk	rF|jj|�|jdrbtj|_ntj	|_|jj
d�|_g|_t
jd�|_g|_d|_tj|jdtjd|j�|_|j�\|_|_g|_tj�|_g|_g|_xzt|j|j�D]c}|j rQ|jj!|�q/||jkrs|jj"|�q/||jkr/|jj"|�q/Wt#|j�|_$t#|j�|_%dS)z�Initializer.

        Args:
            fixer_names: a list of fixers to import
            options: a dict with configuration.
            explicit: a list of fixers to run even if they are explicit.
        NrerfrdF�convert�logger)&�fixers�explicit�_default_options�copy�optionsr(r
�!python_grammar_no_print_statement�grammarr1�getrf�errors�loggingZ	getLoggerrh�	fixer_log�wroterZDriverr	rg�
get_fixers�	pre_order�
post_order�files�bmZ
BottomMatcher�BMZ
bmi_pre_orderZbmi_post_orderrZ
BM_compatibleZ	add_fixerrr7�bmi_pre_order_heads�bmi_post_order_heads)�selfZfixer_namesrmrjr6rrr�__init__�s<	
								zRefactoringTool.__init__cCs�g}g}x�|jD]}t|iidg�}|jdd�d}|j|j�ru|t|j�d�}|jd�}|jdjdd�|D��}yt	||�}Wn(t
k
r�td	||f��YnX||j|j
�}	|	jr4|jd
k	r4||jkr4|jd|�q|jd|�|	jd
krc|j|	�q|	jdkr�|j|	�qtd|	j��qWtjd�}
|jd|
�|jd|
�||fS)aInspects the options to load the requested patterns and handlers.

        Returns:
          (pre_order, post_order), where pre_order is the list of fixers that
          want a pre-order AST traversal, and post_order is the list that want
          post-order traversal.
        r
r8rN�_�cSsg|]}|j��qSr)�title)r9r+rrrr<�s	z.RefactoringTool.get_fixers.<locals>.<listcomp>zCan't find %s.%sTzSkipping optional fixer: %szAdding transformation: %sZpreZpostzIllegal fixer order: %rZ	run_order�key���)rir�rsplitr�FILE_PREFIX�len�split�CLASS_PREFIX�join�getattr�AttributeErrorrbrmrsrj�log_message�	log_debug�orderr�operator�
attrgetter�sort)r}Zpre_order_fixersZpost_order_fixersZfix_mod_path�modr:�partsZ
class_nameZ	fix_classr6Zkey_funcrrrru�s8#
zRefactoringTool.get_fixerscOs�dS)zCalled when an error occurs.Nr)r}�msg�args�kwdsrrr�	log_errorszRefactoringTool.log_errorcGs$|r||}|jj|�dS)zHook to log a message.N)rh�info)r}r�r�rrrr�s
zRefactoringTool.log_messagecGs$|r||}|jj|�dS)N)rh�debug)r}r�r�rrrr�s
zRefactoringTool.log_debugcCsdS)zTCalled with the old version, new version, and filename of a
        refactored file.Nr)r}�old_text�new_text�filename�equalrrr�print_output!szRefactoringTool.print_outputcCsPxI|D]A}tjj|�r5|j|||�q|j|||�qWdS)z)Refactor a list of files and directories.N)rr�isdir�refactor_dir�
refactor_file)r}�items�write�
doctests_onlyZdir_or_filerrr�refactor&s
zRefactoringTool.refactorc
Cs�tjd}x�tj|�D]�\}}}|jd|�|j�|j�xb|D]Z}|jd�rWtjj|�d|krWtjj||�}	|j	|	||�qWWdd�|D�|dd�<qWdS)z�Descends down a directory and refactor every Python file found.

        Python files are assumed to have a .py extension.

        Files and subdirectories starting with '.' are skipped.
        �pyzDescending into %sr8rcSs%g|]}|jd�s|�qS)r8)r)r9Zdnrrrr<As	z0RefactoringTool.refactor_dir.<locals>.<listcomp>N)
r�extsep�walkr�r�rr�splitextr�r�)
r}Zdir_namer�r�Zpy_ext�dirpathZdirnames�	filenamesr�fullnamerrrr�/s



zRefactoringTool.refactor_dircCs�yt|d�}Wn<tk
rQ}z|jd||�dSWYdd}~XnXztj|j�d}Wd|j�Xt|dd|��}t|j	��|fSWdQRXdS)zG
        Do our best to decode a Python source file correctly.
        �rbzCan't open %s: %sNrr*�encoding)NN)
�open�OSErrorr�r�detect_encodingrR�close�_open_with_encodingrC�read)r}r��f�errr�rrr�_read_python_sourceCsz#RefactoringTool._read_python_sourcecCs|j|�\}}|dkr%dS|d7}|r�|jd|�|j||�}|jsl||kr�|j|||||�q|jd|�ni|j||�}|js�|r�|jr�|jt|�dd	�|d|d|�n|jd|�dS)
zRefactors a file.Nr@zRefactoring doctests in %szNo doctest changes in %srr�r�zNo changes in %sr�)r�r��refactor_docstringrf�processed_file�refactor_string�was_changed�str)r}r�r�r�rBr��output�treerrrr�Ss
zRefactoringTool.refactor_filecCs�t|�}d|kr'tj|j_zby|jj|�}WnEtk
r�}z%|jd||jj	|�dSWYdd}~XnXWd|j|j_X||_
|jd|�|j||�|S)aFRefactor a given input string.

        Args:
            data: a string holding the code to be refactored.
            name: a human-readable name for use in error/log messages.

        Returns:
            An AST corresponding to the refactored input stream; None if
            there were errors during the parse.
        rezCan't parse %s: %s: %sNzRefactoring %s)
rar
rnrroZparse_stringr)r��	__class__r!�future_featuresr��
refactor_tree)r}�datarr^r�r�rrrr�js		zRefactoringTool.refactor_stringcCs�tjj�}|ro|jd�|j|d�}|jsI||kr_|j|d|�q�|jd�nS|j|d�}|js�|r�|jr�|jt	|�d|�n
|jd�dS)NzRefactoring doctests in stdinz<stdin>zNo doctest changes in stdinzNo changes in stdin)
�sys�stdinr�r�r�rfr�r�r�r�)r}r�rBr�r�rrr�refactor_stdin�s
zRefactoringTool.refactor_stdinc
Cs�x-t|j|j�D]}|j||�qW|j|j|j��|j|j|j��|jj|j	��}x�t
|j��rLx�|jjD]�}||kr�||r�||j
dtjjdd�|jr�||j
dtjj�xGt||�D]5}|||kr3||j|�yt|�Wntk
rXwYnX|jrt||jkrtq|j|�}|r|j||�}|dk	r|j|�x6|j�D](}|js�g|_|jj|�q�W|jj|j	��}x9|D]1}	|	|kr(g||	<||	j||	�qWqWq�Wq}Wx-t|j|j�D]}|j||�qcW|jS)a�Refactors a parse tree (modifying the tree in place).

        For compatible patterns the bottom matcher module is
        used. Otherwise the tree is traversed node-to-node for
        matches.

        Args:
            tree: a pytree.Node instance representing the root of the tree
                  to be refactored.
            name: a human-readable name for this tree.

        Returns:
            True if the tree was modified, False otherwise.
        r��reverseTN)rrvrwZ
start_tree�traverse_byr{r|rzZrunZleaves�anyr2rir�r	ZBaseZdepthZkeep_line_orderZ
get_linenor/�remover�
ValueErrorZfixers_applied�match�	transformrArr4Zfinish_treer�)
r}r�rr6Z	match_set�node�results�newZnew_matchesZfxrrrrr��sJ 	

		

%zRefactoringTool.refactor_treecCs}|s
dSxl|D]d}x[||jD]L}|j|�}|r%|j||�}|dk	r%|j|�|}q%WqWdS)aTraverse an AST, applying a set of fixers to each node.

        This is a helper method for refactor_tree().

        Args:
            fixers: a list of fixer instances.
            traversal: a generator that yields AST nodes.

        Returns:
            None
        N)r%r�r�rA)r}riZ	traversalr�r6r�r�rrrr��s

zRefactoringTool.traverse_bycCs�|jj|�|dkr?|j|�d}|dkr?dS||k}|j||||�|r�|jd|�|js�dS|r�|j||||�n|jd|�dS)zR
        Called when a file has been refactored and there may be changes.
        NrzNo changes to %szNot writing changes to %s)rxrr�r�r�rf�
write_file)r}r�r�r�r�r�r�rrrr��s	zRefactoringTool.processed_filec%Cs�yt|dd|�}Wn<tk
rW}z|jd||�dSWYdd}~XnXzVy|jt|��Wn8tk
r�}z|jd||�WYdd}~XnXWd|j�X|jd|�d|_dS)z�Writes a string to a file.

        It first shows a unified diff between the old text and the new text, and
        then rewrites the file; the latter is only done if the write option is
        set.
        �wr�zCan't create %s: %sNzCan't write %s: %szWrote changes to %sT)r�r�r�r�rEr�r�rt)r}r�r�r�r�r�r�rrrr�s*zRefactoringTool.write_filez>>> z... c
Cs�g}d}d}d}d}x.|jdd�D]}|d7}|j�j|j�r�|dk	r�|j|j||||��|}|g}|j|j�}	|d|	�}q1|dk	r|j||j�s�|||jj�dkr|j	|�q1|dk	r2|j|j||||��d}d}|j	|�q1W|dk	rz|j|j||||��dj
|�S)a�Refactors a docstring, looking for doctests.

        This returns a modified version of the input string.  It looks
        for doctests, which start with a ">>>" prompt, and may be
        continued with "..." prompts, as long as the "..." is indented
        the same as the ">>>".

        (Unfortunately we can't use the doctest module's parser,
        since, like most parsers, it is not geared towards preserving
        the original source.)
        Nr�keependsTrr@r�)�
splitlines�lstripr�PS1r4�refactor_doctest�find�PS2�rstriprr�)
r}rBr��result�blockZblock_lineno�indent�lineno�line�irrrr�(s:

	

z"RefactoringTool.refactor_docstringc
s�y�j||��}Wn�tk
r�}zg�jjtj�rmx'|D]}�jd|jd��qJW�jd|||j	j
|�|SWYdd}~XnX�j||�r}t|�j
dd�}|d|d�||dd�}	}|	dg|dkst|	��|djd�s:|dd7<��j|jd�g}|r}|��fd	d
�|D�7}|S)
z�Refactors one doctest.

        A doctest is given as a block of lines, the first of which starts
        with ">>>" (possibly indented), while the remaining lines start
        with "..." (identically indented).

        z
Source: %sr@z+Can't parse docstring in %s line %s: %s: %sNr�Trrcs!g|]}��j|�qSr)r�)r9r�)r�r}rrr<ms	z4RefactoringTool.refactor_doctest.<locals>.<listcomp>r�r�)�parse_blockr)rhZisEnabledForrr�DEBUGr�r�r�r�r!r�r�r��AssertionErrorrr��pop)
r}r�r�r�r�r�r�r�r�Zclippedr)r�r}rr�Ss&
	)# z RefactoringTool.refactor_doctestcCs|jrd}nd}|js4|jd|�n1|jd|�x|jD]}|j|�qNW|jr�|jd�x|jD]}|j|�q�W|jrt|j�dkr�|jd�n|jdt|j��x-|jD]"\}}}|j|||�q�WdS)	N�werez
need to bezNo files %s modified.zFiles that %s modified:z$Warnings/messages while refactoring:rzThere was 1 error:zThere were %d errors:)rtrxr�rsrqr�)r}r��file�messager�r�r�rrr�	summarizeps$				
	zRefactoringTool.summarizecCs1|jj|j|||��}t�|_|S)z�Parses a block into a tree.

        This is necessary to get correct line number / offset information
        in the parser diagnostics and embedded into the parse tree.
        )rZparse_tokens�	wrap_toksrSr�)r}r�r�r�r�rrrr��s!zRefactoringTool.parse_blockccs�tj|j||�j�}xe|D]]\}}\}}\}	}
}||d7}|	|d7}	||||f|	|
f|fVq%WdS)z;Wraps a tokenize stream to systematically modify start/end.rN)rrO�	gen_lines�__next__)r}r�r�r�r3r%r`Zline0Zcol0Zline1Zcol1Z	line_textrrrr��s
(zRefactoringTool.wrap_toksccs�||j}||j}|}xo|D]g}|j|�rT|t|�d�Vn4||j�dkrrdVntd||f��|}q'Wx	dVq�WdS)z�Generates lines as expected by tokenize from a list of lines.

        This strips the first len(indent + self.PS1) characters off each line.
        Nr@zline=%r, prefix=%rr�)r�r�rr�r�r�)r}r�r�Zprefix1Zprefix2�prefixr�rrrr��s



zRefactoringTool.gen_lines)r!r"r#rkr�r�r~rur�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�r�rrrrrd�s:4(	O+
rdc@seZdZdS)�MultiprocessingUnsupportedN)r!r"r#rrrrr��sr�csaeZdZ�fdd�Zddd�fdd�Z�fdd�Z�fd	d
�Z�S)�MultiprocessRefactoringToolcs/tt|�j||�d|_d|_dS)N)�superr�r~�queue�output_lock)r}r��kwargs)r�rrr~�s	z$MultiprocessRefactoringTool.__init__FrcsU|dkr(tt��j|||�Syddl�Wntk
rRt�YnX�jdk	rntd���j��_�j	��_
��fdd�t|�D�}z;x|D]}|j�q�Wtt��j|||�Wd�jj
�x$t|�D]}�jjd�qWx$|D]}|j�r'|j
�q'Wd�_XdS)Nrrz already doing multiple processescs%g|]}�jd�j��qS)�target)ZProcess�_child)r9r�)�multiprocessingr}rrr<�s	z8MultiprocessRefactoringTool.refactor.<locals>.<listcomp>)r�r�r�r��ImportErrorr�r��RuntimeErrorZ
JoinableQueueZLockr��range�startr��putZis_alive)r}r�r�r�Z
num_processesZ	processesr+r�)r�)r�r}rr��s2




z$MultiprocessRefactoringTool.refactorc
so|jj�}xY|dk	rj|\}}ztt|�j||�Wd|jj�X|jj�}qWdS)N)r�rpr�r�r�Z	task_done)r}Ztaskr�r�)r�rrr��sz"MultiprocessRefactoringTool._childcsE|jdk	r(|jj||f�ntt|�j||�SdS)N)r�r�r�r�r�)r}r�r�)r�rrr��sz)MultiprocessRefactoringTool.refactor_file)r!r"r#r~r�r�r�rr)r�rr��s
r�),rcrKr�
__author__rr�rrr�r-rP�	itertoolsrZpgen2rrrZ
fixer_utilrr�r	r
rZburryrr)r r&r7r=r?�version_info�codecsr�r�rCrErarb�objectrdr�r�rrrr�<module>	sF
	(��