HEX
Server: Apache
System: Linux aura.ambarhosting.com 4.18.0-553.lve.el8.x86_64 #1 SMP Mon May 27 15:27:34 UTC 2024 x86_64
User: chicagos (1295)
PHP: 8.1.33
Disabled: NONE
Upload Files
File: //lib/python2.7/site-packages/s3transfer/utils.pyc
�
P$�_c@svddlZddlZddlZddlZddlZddlZddlZddlZddlZddl	Z	ddl
Z
ddlmZddl
mZddl
mZddlmZddlmZddlmZddlmZd	Zd
d-Zd
d.Zeje�ZejeeefZdd�Zd�Zd�Zd�Z dd�Z"d�Z#d�Z$d�Z%de&fd��YZ'de&fd��YZ(de&fd��YZ)de&fd��YZ*de&fd ��YZ+d!e&fd"��YZ,d#e&fd$��YZ-d%e.fd&��YZ/d'e&fd(��YZ0d)e0fd*��YZ1d+e&fd,��YZ2dS(/i����N(tdefaultdict(tIncompleteReadError(tReadTimeoutError(tSOCKET_ERROR(trename_file(tseekable(t	fallocatei'iiiiicCsdjd�t|�D��S(Ntcss!|]}tjtj�VqdS(N(trandomtchoicetstringt	hexdigits(t.0t_((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pys	<genexpr>2s(tjointrange(t
num_digits((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pytrandom_file_extension1scKs2|dkr.t|jd�r.|jj�ndS(Nt	PutObjectt
UploadParttsignal_not_transferring(RR(thasattrtbodyR(trequesttoperation_nametkwargs((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR5scKs2|dkr.t|jd�r.|jj�ndS(NRRtsignal_transferring(s	PutObjects
UploadPart(RRR(RRR((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR;scCsttj|t|���S(N(tinttmathtceiltfloat(tsizet	part_size((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pytcalculate_num_partsAscCsd||}||dkrBd}|dk	rPt|d�}qPn||d}d||f}|S(s�Calculate the range parameter for multipart downloads/copies

    :type part_size: int
    :param part_size: The size of the part

    :type part_index: int
    :param part_index: The index for which this parts starts. This index starts
        at zero

    :type num_parts: int
    :param num_parts: The total number of parts in the transfer

    :returns: The value to use for Range parameter on downloads or
        the CopySourceRange parameter for copies
    iRsbytes=%s-%sN(tNonetstr(R t
part_indext	num_partst
total_sizetstart_ranget	end_rangetrange_param((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pytcalculate_range_parameterEs
cCseg}xX|jjjD]G}d|}t||�r|jtjt||�d|��qqW|S(sRetrieves callbacks from a subscriber

    :type transfer_future: s3transfer.futures.TransferFuture
    :param transfer_future: The transfer future the subscriber is associated
        to.

    :type callback_type: str
    :param callback_type: The type of callback to retrieve from the subscriber.
        Valid types include:
            * 'queued'
            * 'progress'
            * 'done'

    :returns: A list of callbacks for the type specified. All callbacks are
        preinjected with the transfer future.
    ton_tfuture(tmetat	call_argstsubscribersRtappendt	functoolstpartialtgetattr(ttransfer_futuret
callback_typet	callbackst
subscribert
callback_name((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyt
get_callbacksbs
cCs+|r'x|D]}|d|�q
WndS(s�Calls all progress callbacks

    :param callbacks: A list of progress callbacks to invoke
    :param bytes_transferred: The number of bytes transferred. This is passed
        to the callbacks. If no bytes were transferred the callbacks will not
        be invoked because no progress was achieved. It is also possible
        to receive a negative amount which comes from retrying a transfer
        request.
    tbytes_transferredN((R6R:tcallback((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pytinvoke_progress_callbacks�s
cCs@i}x3|j�D]%\}}||kr|||<qqW|S(soGets a dictionary filtered by whitelisted keys

    :param original_dict: The original dictionary of arguments to source keys
        and values.
    :param whitelisted_key: A list of keys to include in the filtered
        dictionary.

    :returns: A dictionary containing key/values from the original dictionary
        whose key was included in the whitelist
    (titems(t
original_dicttwhitelisted_keyst
filtered_dicttkeytvalue((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pytget_filtered_dict�s
tCallArgscBseZd�ZRS(cKs1x*|j�D]\}}t|||�q
WdS(s�A class that records call arguments

        The call arguments must be passed as keyword arguments. It will set
        each keyword argument as an attribute of the object along with its
        associated value.
        N(R=tsetattr(tselfRtargRB((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyt__init__�s(t__name__t
__module__RH(((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyRD�stFunctionContainercBs)eZdZd�Zd�Zd�ZRS(s�An object that contains a function and any args or kwargs to call it

    When called the provided function will be called with provided args
    and kwargs.
    cOs||_||_||_dS(N(t_funct_argst_kwargs(RFtfunctargsR((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyRH�s		cCsd|j|j|jfS(Ns'Function: %s with args %s and kwargs %s(RLRMRN(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyt__repr__�scCs|j|j|j�S(N(RLRMRN(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyt__call__�s(RIRJt__doc__RHRQRR(((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyRK�s		tCountCallbackInvokercBsAeZdZd�Zed��Zd�Zd�Zd�ZRS(s�An abstraction to invoke a callback when a shared count reaches zero

    :param callback: Callback invoke when finalized count reaches zero
    cCs.tj�|_||_d|_t|_dS(Ni(t	threadingtLockt_lockt	_callbackt_counttFalset
_is_finalized(RFR;((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyRH�s		cCs|j�|jSWdQXdS(N(RWRY(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyt
current_count�s
cCs;|j�,|jr"td��n|jd7_WdQXdS(sIncrement the count by ones;Counter has been finalized it can no longer be incremented.iN(RWR[tRuntimeErrorRY(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyt	increment�s

	cCsf|j�W|jdkr(td��n|jd8_|jr\|jdkr\|j�nWdQXdS(sDecrement the count by oneis,Counter is at zero. It cannot dip below zeroiN(RWRYR]R[RX(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyt	decrement�s
cCs9|j�*t|_|jdkr/|j�nWdQXdS(s�Finalize the counter

        Once finalized, the counter never be incremented and the callback
        can be invoked once the count reaches zero
        iN(RWtTrueR[RYRX(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pytfinalize�s
	(	RIRJRSRHtpropertyR\R^R_Ra(((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyRT�s		
	
tOSUtilscBsbeZdZd�Zd�Zd
d�Zd�Zd�Zd�Z	d�Z
d�Zd	�ZRS(i�cCstjj|�S(N(tostpathtgetsize(RFtfilename((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyt
get_file_size�scCstj||||dt�S(Ntenable_callbacks(t
ReadFileChunkt
from_filenameRZ(RFRgt
start_byteRR6((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pytopen_file_chunk_reader�s	c
Cs"t|||d|dtd|�S(NR6Ritclose_callbacks(RjRZ(RFtfileobjt
chunk_sizetfull_file_sizeR6Rn((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyt#open_file_chunk_reader_from_fileobj�scCs
t||�S(N(topen(RFRgtmode((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyRsscCs)ytj|�Wntk
r$nXdS(s+Remove a file, noop if file does not exist.N(RdtremovetOSError(RFRg((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pytremove_files
cCst||�dS(N(R(RFtcurrent_filenametnew_filename((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyRscCsxtjj|�stStj|�j}tj|�r;tStj|�rNtStj	|�ratStj
|�rttStS(sChecks to see if a file is a special UNIX file.

        It checks if the file is a character special device, block special
        device, FIFO, or socket.

        :param filename: Name of the file

        :returns: True if the file is a special file. False, if is not.
        (RdRetexistsRZtstattst_modetS_ISCHRR`tS_ISBLKtS_ISFIFOtS_ISSOCK(tclsRgRt((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pytis_special_filescCsbtjt�}tjj|�}tjj|�}||jt|� |}tjj||�S(N(	RdtextsepRRetdirnametbasenamet_MAX_FILENAME_LENtlenR(RFRgtsuffixRetnamet
temp_filename((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pytget_temp_filename0s
cCsZy,|j|d��}t||�WdQXWn'ttfk
rU|j|��nXdS(Ntwb(RsRRvtIOErrorRw(RFRgRtf((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pytallocate7s
N(
RIRJR�RhRmR"RrRsRwRR�R�R�(((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyRc�s								tDeferredOpenFilecBsteZdded�Zd�Zed��Zdd�Zd�Z	d�Z
d�Zd	�Zd
�Z
d�ZRS(
itrbcCs1||_d|_||_||_||_dS(s�A class that defers the opening of a file till needed

        This is useful for deferring opening of a file till it is needed
        in a separate thread, as there is a limit of how many open files
        there can be in a single thread for most operating systems. The
        file gets opened in the following methods: ``read()``, ``seek()``,
        and ``__enter__()``

        :type filename: str
        :param filename: The name of the file to open

        :type start_byte: int
        :param start_byte: The byte to seek to when the file is opened.

        :type mode: str
        :param mode: The mode to use to open the file

        :type open_function: function
        :param open_function: The function to use to open the file
        N(t	_filenameR"t_fileobjt_start_bytet_modet_open_function(RFRgRlRtt
open_function((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyRHAs
				cCsV|jdkrR|j|j|j�|_|jdkrR|jj|j�qRndS(Ni(R�R"R�R�R�R�tseek(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyt_open_if_needed\scCs|jS(N(R�(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�bscCs|j�|jj|�S(N(R�R�tread(RFtamount((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�fs
cCs|j�|jj|�dS(N(R�R�twrite(RFtdata((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�js
cCs|j�|jj|�dS(N(R�R�R�(RFtwhere((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�ns
cCs#|jdkr|jS|jj�S(N(R�R"R�ttell(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�rscCs|jr|jj�ndS(N(R�tclose(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�ws	cCs|j�|S(N(R�(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyt	__enter__{s
cOs|j�dS(N(R�(RFRPR((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyt__exit__sN(RIRJRsRHR�RbR�R"R�R�R�R�R�R�R�(((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�@s						RjcBs�eZdedd�Zeded��Zd�Zdd�Zd�Z	d�Z
d�Zd�Zd�Z
d	�Zd
�Zd�Zd�Zd
�Zd�ZRS(cCs�||_|jj�|_|j|jd|d|jd|�|_d|_||_|dkrog|_n||_||_	|dkr�||_	ndS(s�

        Given a file object shown below::

            |___________________________________________________|
            0          |                 |                 full_file_size
                       |----chunk_size---|
                    f.tell()

        :type fileobj: file
        :param fileobj: File like object

        :type chunk_size: int
        :param chunk_size: The max chunk size to read.  Trying to read
            pass the end of the chunk size will behave like you've
            reached the end of the file.

        :type full_file_size: int
        :param full_file_size: The entire content length associated
            with ``fileobj``.

        :type callbacks: A list of function(amount_read)
        :param callbacks: Called whenever data is read from this object in the
            order provided.

        :type enable_callbacks: boolean
        :param enable_callbacks: True if to run callbacks. Otherwise, do not
            run callbacks

        :type close_callbacks: A list of function()
        :param close_callbacks: Called when close is called. The function
            should take no arguments.
        trequested_sizeRltactual_file_sizeiN(
R�R�R�t_calculate_file_sizet_sizet_amount_readt
_callbacksR"t_callbacks_enabledt_close_callbacks(RFRoRpRqR6RiRn((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyRH�s#					cCsJt|d�}|j|�tj|j��j}||||||�S(s[Convenience factory function to create from a filename.

        :type start_byte: int
        :param start_byte: The first byte from which to start reading.

        :type chunk_size: int
        :param chunk_size: The max chunk size to read.  Trying to read
            pass the end of the chunk size will behave like you've
            reached the end of the file.

        :type full_file_size: int
        :param full_file_size: The entire content length associated
            with ``fileobj``.

        :type callbacks: function(amount_read)
        :param callbacks: Called whenever data is read from this object.

        :type enable_callbacks: bool
        :param enable_callbacks: Indicate whether to invoke callback
            during read() calls.

        :rtype: ``ReadFileChunk``
        :return: A new instance of ``ReadFileChunk``

        R�(RsR�Rdtfstattfilenotst_size(R�RgRlRpR6RiR�t	file_size((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyRk�s
cCs||}t||�S(N(tmin(RFRoR�RlR�tmax_chunk_size((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR��s
cCs�|dkr|j|j}nt|j|j|�}|jj|�}|jt|�7_|jdk	r�|jr�t	|jt|��n|S(N(
R"R�R�R�R�R�R�R�R�R<(RFR�tamount_to_readR�((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR��scCs0|j�t|jd�r,|jj�ndS(NR(tenable_callbackRR�R(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�s
cCs0|j�t|jd�r,|jj�ndS(NR(tdisable_callbackRR�R(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�s
cCs
t|_dS(N(R`R�(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR��scCs
t|_dS(N(RZR�(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR��scCsY|jj|j|�|jdk	rL|jrLt|jd||j�n||_dS(NR:(R�R�R�R�R"R�R<R�(RFR�((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR��s
cCsG|jdk	r6|jr6x|jD]
}|�q"Wn|jj�dS(N(R�R"R�R�R�(RFR;((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR��scCs|jS(N(R�(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�scCs|jS(N(R�(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyt__len__scCs|S(N((RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�scOs|j�dS(N(R�(RFRPR((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�scCs
tg�S(N(titer(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyt__iter__sN(RIRJR"R`RHtclassmethodRkR�R�RRR�R�R�R�R�R�R�R�R�(((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyRj�s"0											tStreamReaderProgresscBs#eZdZdd�Zd�ZRS(s<Wrapper for a read only stream that adds progress callbacks.cCs.||_||_|dkr*g|_ndS(N(t_streamR�R"(RFtstreamR6((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyRH s		cOs/|jj||�}t|jt|��|S(N(R�R�R<R�R�(RFRPRRB((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�&sN(RIRJRSR"RHR�(((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�stNoResourcesAvailablecBseZRS((RIRJ(((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�,st
TaskSemaphorecBs&eZd�Zed�Zd�ZRS(cCstj|�|_dS(sqA semaphore for the purpose of limiting the number of tasks

        :param count: The size of semaphore
        N(RUt	Semaphoret
_semaphore(RFtcount((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyRH1scCs9tjd|�|jj|�s5td|��ndS(s�Acquire the semaphore

        :param tag: A tag identifying what is acquiring the semaphore. Note
            that this is not really needed to directly use this class but is
            needed for API compatibility with the SlidingWindowSemaphore
            implementation.
        :param block: If True, block until it can be acquired. If False,
            do not block and raise an exception if cannot be aquired.

        :returns: A token (can be None) to use when releasing the semaphore
        sAcquiring %ssCannot acquire tag '%s'N(tloggertdebugR�tacquireR�(RFttagtblocking((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�8scCs(tjd||f�|jj�dS(stRelease the semaphore

        :param tag: A tag identifying what is releasing the semaphore
        :param acquire_token:  The token returned from when the semaphore was
            acquired. Note that this is not really needed to directly use this
            class but is needed for API compatibility with the
            SlidingWindowSemaphore implementation.
        sReleasing acquire %s/%sN(R�R�R�trelease(RFR�t
acquire_token((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�Hs	(RIRJRHR`R�R�(((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�0s	tSlidingWindowSemaphorecBs5eZdZd�Zd�Zed�Zd�ZRS(s�A semaphore used to coordinate sequential resource access.

    This class is similar to the stdlib BoundedSemaphore:

    * It's initialized with a count.
    * Each call to ``acquire()`` decrements the counter.
    * If the count is at zero, then ``acquire()`` will either block until the
      count increases, or if ``blocking=False``, then it will raise
      a NoResourcesAvailable exception indicating that it failed to acquire the
      semaphore.

    The main difference is that this semaphore is used to limit
    access to a resource that requires sequential access.  For example,
    if I want to access resource R that has 20 subresources R_0 - R_19,
    this semaphore can also enforce that you only have a max range of
    10 at any given point in time.  You must also specify a tag name
    when you acquire the semaphore.  The sliding window semantics apply
    on a per tag basis.  The internal count will only be incremented
    when the minimum sequence number for a tag is released.

    cCsR||_tt�|_i|_tj�|_tj|j�|_	i|_
dS(N(RYRRt_tag_sequencest_lowest_sequenceRURVRWt	Conditiont
_conditiont_pending_release(RFR�((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyRHks		cCs|j�|jSWdQXdS(N(RWRY(RF((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR\us
cCs�tjd|�|jj�z�|jdkrn|sHtd|��qnx#|jdkrj|jj�qKWn|j|}|dkr�||j|<n|j|cd7<|jd8_|SWd|jj	�XdS(NsAcquiring %sisCannot acquire tag '%s'i(
R�R�R�R�RYR�twaitR�R�R�(RFR�R�tsequence_number((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�ys

cCs�|}tjd||�|jj�zP||jkrKtd|��n|j|}|j||kr|j|cd7<|jd7_|jj�|j	j
|g�}x�|r|j||dkr�|j�|j|cd7<|jd7_q�Pq�Wno|j||ko$|knr_|j	j|g�j
|�|j	|jdt�ntd||f��Wd|jj�XdS(NsReleasing acquire %s/%ss$Attempted to release unknown tag: %sii����treverses;Attempted to release unknown sequence number %s for tag: %s(R�R�R�R�R�t
ValueErrorR�RYtnotifyR�tgettpopt
setdefaultR0tsortR`R�(RFR�R�R�tmax_sequencetqueued((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR��s4


	
#	(RIRJRSRHR\R`R�R�(((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR�Us
	
	tChunksizeAdjustercBs8eZeeed�Zdd�Zd�Zd�Z	RS(cCs||_||_||_dS(N(tmax_sizetmin_sizet	max_parts(RFR�R�R�((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyRH�s		cCs4|}|dk	r'|j||�}n|j|�S(s�Get a chunksize close to current that fits within all S3 limits.

        :type current_chunksize: int
        :param current_chunksize: The currently configured chunksize.

        :type file_size: int or None
        :param file_size: The size of the file to upload. This might be None
            if the object being transferred has an unknown size.

        :returns: A valid chunksize that fits within configured limits.
        N(R"t_adjust_for_max_partst_adjust_for_chunksize_limits(RFtcurrent_chunksizeR�t	chunksize((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pytadjust_chunksize�scCsh||jkr0tjd|j|f�|jS||jkr`tjd|j|f�|jS|SdS(Ns@Chunksize greater than maximum chunksize. Setting to %s from %s.s=Chunksize less than minimum chunksize. Setting to %s from %s.(R�R�R�R�(RFR�((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR��scCs�|}ttj|t|���}x<||jkrc|d9}ttj|t|���}q(W||kr�tjd||f�n|S(Nis[Chunksize would result in the number of parts exceeding the maximum. Setting to %s from %s.(RRRRR�R�R�(RFR�R�R�R%((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR��s
#N(
RIRJtMAX_SINGLE_UPLOAD_SIZEtMIN_UPLOAD_CHUNKSIZEt	MAX_PARTSRHR"R�R�R�(((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyR��s
	i@i(3RttimeR1RRdtsocketR{R
tloggingRUtiotcollectionsRtbotocore.exceptionsRRts3transfer.compatRRRRR�R�R�t	getLoggerRIR�ttimeouttS3_RETRYABLE_DOWNLOAD_ERRORSRRRR!R"R*R9R<RCtobjectRDRKRTRcR�RjR�t	ExceptionR�R�R�R�(((s4/usr/lib/python2.7/site-packages/s3transfer/utils.pyt<module>
sR

						0OC�%_