o ]Lb@s$dZddlmZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z ddl mZddlmZmZmZmZmZmZmZmZmZmZmZddlmZdd lmZmZd d l m!Z!m"Z"m#Z#d Z$iZ%e&e%Z&e&d ddde&d ddde&d ddde&d de'de&d ddde&d dddde&ddddddgZ(dd Z)d!d"Z*d#d$Z+e!j,Z-e.pe/Z0e1e-e1e0kZ2d%d&Z3d@d'd(Z4 )      dAd*d+Z5Gd,d-d-e6Z7d.d/Z8d0d1Z9d2d3Z:d4d5Z;Gd6d7d7e6Zdd?Z@dS)Cab Faster status operations with the Watchman file monitor (EXPERIMENTAL) Integrates the file-watching program Watchman with Mercurial to produce faster status results. On a particular Linux system, for a real-world repository with over 400,000 files hosted on ext4, vanilla `hg status` takes 1.3 seconds. On the same system, with fsmonitor it takes about 0.3 seconds. fsmonitor requires no configuration -- it will tell Watchman about your repository as necessary. You'll need to install Watchman from https://facebook.github.io/watchman/ and make sure it is in your PATH. fsmonitor is incompatible with the largefiles and eol extensions, and will disable itself if any of those are active. The following configuration options exist: :: [fsmonitor] mode = {off, on, paranoid} When `mode = off`, fsmonitor will disable itself (similar to not loading the extension at all). When `mode = on`, fsmonitor will be enabled (the default). When `mode = paranoid`, fsmonitor will query both Watchman and the filesystem, and ensure that the results are consistent. :: [fsmonitor] timeout = (float) A value, in seconds, that determines how long fsmonitor will wait for Watchman to return results. Defaults to `2.0`. :: [fsmonitor] blacklistusers = (list of userids) A list of usernames for which fsmonitor will disable itself altogether. :: [fsmonitor] walk_on_invalidate = (boolean) Whether or not to walk the whole repo ourselves when our cached state has been invalidated, for example when Watchman has been restarted or .hgignore rules have been changed. Walking the repo in that case can result in competing for I/O with Watchman. For large repos it is recommended to set this value to false. You may wish to set this to true if you have a very fast filesystem that can outpace the IPC overhead of getting the result data for the full repo from Watchman. Defaults to false. :: [fsmonitor] warn_when_unused = (boolean) Whether to print a warning during certain operations when fsmonitor would be beneficial to performance but isn't enabled. :: [fsmonitor] warn_update_file_count = (integer) # or when mercurial is built with rust support warn_update_file_count_rust = (integer) If ``warn_when_unused`` is set and fsmonitor isn't enabled, a warning will be printed during working directory updates if this many files will be created. )absolute_importN)_)hex)open) contextencodingerror extensions localrepomergepathutilpycompat registrarscmutilutil)match)hashutil stringutil) pywatchmanstatewatchmanclientsships-with-hg-core fsmonitormodeson)defaultswalk_on_invalidateFstimeout2sblacklistusers watchman_exewatchmanverboseT)r experimental experimentalfsmonitor.transaction_notifys largefilesseolc Cs|dtd|ddt}t||}d}z|d}|dtdt |dWntj yF}z t |}WYd}~nd}~ww| |d td ||rUd Sd S) Nsfsmonitor-watchmans/fsmonitor checking for watchman binary... (%s) rrsversionsfsmonitor-watchman-versions watchman binary version %s versionsfsmonitor-watchman-errors' watchman binary missing or broken: %s rr)writer configpathtempfilemkdtemprclientcommandr bytestr Unavailabler forcebytestr condwrite)uifmrootcerrver4:/usr/lib/python3/dist-packages/hgext/fsmonitor/__init__.py debuginstalls4     r6cCst|tjr<|jr |ddr dt|vr |t|d|jr'||ddr:|ddt|j dSdS|ddt|dS)z5Exception handler for Watchman interaction exceptionsrrsillegal_fstypes sWatchman unavailable: %s sWatchman exception: %s N) isinstancerr*warn configboolrr+ invalidatelogmsg)r-rexr4r4r5_handleunavailables$   r?cCs&t}|t|t|S)zCalculate hash for ignore patterns and filenames If this information changes between Mercurial invocations, we can't rely on Watchman information anymore and have to re-scan the working copy. )rsha1updater bytereprsysbytes hexdigest)ignorer@r4r4r5 _hashignoresrFc Csxz|t}Wnty}z tjt|ddd}~wwz |td}W|St y;}ztt|d}~ww)zFix path to match watchman and local filesystem encoding watchman's paths encoding can differ from filesystem encoding. For example, on Windows, it's always utf-8. swatchman encoding error)hintNstrict) decode_watchmanencodingUnicodeDecodeErrorrAbortrr+encode _fsencodingUnicodeEncodeError)pathdecodedr3encodedr4r4r5_watchmantofsencoding s rSc( s fdd}|r|dSr|dS js |dS j}|\} } } | s7|jr3|dSd} g} j jrUt| krT| dkrT|jrP|dSg} d} ntj tj j  } j t d rnj d d j D} j j}tj}tj}tj}tj} jtj d }d }}rd }tj nrd }|s jr j d }nd \ }}|o|p| }fdd|D}|s|s|r݈D]} |=q҈ d= Sz) j|jd jdgd| ddddggdggt|jd|jd}Wn#ty)}zt j!|| j"|dWYd}~Sd}~ww|#t$%|d|drI|jrE|&|dSd }g} rU fdd  Dt$j'd!k}|d"D]}|d#}t(|t)so|*t+}t,rvt-|}|r|.d!d$} r |} |d d }||<|d%}|d&} ||}!d'|vs|/d(r|d)S| s| vr|vr| sˆ|rd |<q^|!|kr|vr| sڈ|rd |<q^|!|ks|!|kr|vr| s|r| |<q^| s|r|s| |<q^|vr!| s|r!d |<q^ r1 fd*d | D}  fd+d | D}"|sw| r\|"0 fd,d-| D|"0 fd.d-|DnW|"0 fd/d-| D|"0 fd0d-|Dn<| r|"0 fd1d-t$1D|"0 fd2d-|Dn|"0 fd3d-t$1D|"0 fd4d-|Dt2j3 j4d d5j5fd6d|"D}#|#6|"7|#}$|$D]}%d |%<qt8|#}&t9fd7d|#DD]}'t:|&}%|'s|%vr|' |%<qD]} |=q d= S)8zReplacement for dirstate.walk, hooking into Watchman. Whenever full is False, ignored is False, and the Watchman client is available, use Watchman combined with saved state to possibly return only a subset of files.cs"jd|ddS)Ns'fsmonitor: fallback to core status, %s T)full)_uidebug)reason)ignoredrorigselfsubreposunknownr4r5bail'szoverridewalk..bailsfull rewalk requestedslisting ignored filessclient unavailablesno clocksc:0:0signore rules changeds_mapcSs,h|]\}}|dks|dkr|qS)n)v1_statev1_mtime).0fr3r4r4r5 Ps zoverridewalk..FTNcsg|] }|ds|qS)rr4)rbd) dirignorer4r5 ssz overridewalk...hg皙?squery)rsmtimessizeexistsnamesnotsanyofsdirname)rkrhs wholenamei)sfieldsssinces expressions sync_timeoutsempty_on_fresh_instancesexception during runsclocksis_fresh_instancesfresh instancecsi|]}||qSr4r4)rbk)normcaser4r5 sz overridewalk..\sfilesrk/rrjs/.hg/s/.hgsnested-repo-detectedcs$h|]}|vr|ddqSTr4rbrc)foldmap normalizermr4r5rds   cs0h|]}|vr|r|vs|s|qSr4r4rr)dmaprEmatchfnresultsr4r5rds  c3|] }|vr|VqdSNr4rrrwr4r5 zoverridewalk..c3rxryr4rrrzr4r5r{r|c3$|] }|vr|r|VqdSryr4rrrvrwr4r5r{sc3r}ryr4rrr~r4r5r{"c3s |] \}}|vr|VqdSryr4rbrcstrzr4r5r{sc3rxryr4rrrzr4r5r{r|c3s(|]\}}|vr|r|VqdSryr4rr~r4r5r{sc3r}ryr4rrr~r4r5r{r)cachedcsg|]}|r|qSr4r4rr)auditr4r5rgscsg|]}|qSr4r4rr)joinr4r5rg s);_watchmanclient available_fsmonitorstategetwalk_on_invalidate_ignore _dirignorerFralwaysrv_map safehasattritemscopymapstatS_IFMTS_IFDIRS_IFREGS_IFLNK_joinrmisexactprefix _checkcase_normalizefile _walkexplicit settimeouttimeoutr(int Exceptionr?rUclearconnection setlastclockr rCr;ossepr8bytesrMrJ _fixencodingrSreplaceendswithrA iteritemsr pathauditor_rootchecksort differenceiter statfilesnext)(rYrZrr[r\rXrTr]rclock ignorehash notefiles matchalways nonnormalsetrgetkinddirkindregkindlnkkindfresh_instanceexact skipstep3work dirsnotfoundsresultr>switch_slashesentryfnamenormedfmodefexistskindvisit auditpass auditfailrcnfrr4)rrfrursrErXrrrvrtrmrYrwrZr[r\r5 overridewalk sX                     r.c s|} |} |} fdd} t|tjr|} n|} t|tjr"|}n|}|du}|o3| dk}|p9t}|oN|oNt|tjtjf oNdtj v}zj j r[j dn j j jdj }Wn)ty}zj tjj |||||| | | |WYd}~Sd}~ww|rd}n| }|rt|}|||||| | ||}|\}}}}}}}| sg}| p|jdu}j jdkrJ|sJdj_jj}dj_jjjj}}ttjdj_j_z||||| | | |}Wd j_|j_||j_j_nd j_|j_||j_j_w | |||||||g|Wdn 1s._cmpsetsrs HG_PENDINGriTsparanoidrF)#r8r changectxrevmatchmodrworkingcommitctxmemctxrrrrrrrgetcurrentclockrrr?r- poststatusaddpostdsstatus traversedirmodedirstate_fsmonitordisablequietfoutferrrosdevnullwlockrstatus) rYrZnode1node2rrXcleanr\ listsubrepos listignored listclean listunknownrctx1ctx2working parentworking updatestate startclockr> stateunknownpsrmodifiedaddedremoveddeletedrTrrrrv2r4rr5overridestatuss           rc@seZdZddZddZdS)rcCst||_dSry)r rC _startclock)rZrr4r4r5__init__szpoststatus.__init__cCsZ|jp |j}t|jj}|j|j|j |j |j }|j |||dSry) repor getlastclockrrFrrrrrrr\r)rZwctxrr hashignorerr4r4r5__call__szpoststatus.__call__N)__name__ __module__ __qualname__rr r4r4r4r5rs rcs*Gfddd|j|_||dS)NcsBeZdZddZfddZfddZfddZZS) z'makedirstate..fsmonitordirstatecSs&d|_|j|_|j|_t||_dSNF)rrrweakrefproxy_repo)rZr r4r4r5_fsmonitorinitsz6makedirstate..fsmonitordirstate._fsmonitorinitcs8t|j}|jr||i|St||g|Ri|Sry)superwalkrrrZargskwargsrY __class__fsmonitordirstater4r5rs z,makedirstate..fsmonitordirstate.walkcs |jt|j|i|Sry)rr;rrebuildrZrrrr4r5r z/makedirstate..fsmonitordirstate.rebuildcs |jt|j|i|Sry)rr;rrrr4r5r;r z2makedirstate..fsmonitordirstate.invalidate)rrrrrrr; __classcell__r4rrr5rs r)rr)r rr4r"r5 makedirstatesr$cCs"||}t|drt|||S)Ns_fsmonitorstate)rrr$)rYrZdsr4r4r5 wrapdirstates  r&cCs6ttjdttjrttdt tt dt dS)Ndirstatessymlinks_update) r wrapfilecacher localrepositoryr&r isdarwin wrapfunctionr wrapsymlinkr wrapupdate)r-r4r4r5extsetups r.cCsdz|||Wz ttj|dWStyYSwz ttj|dWwty1Yww)zyif we create a dangling symlink, also touch the parent dir to encourage fsevents notifications to work more correctlyN)rutimerPdirnamer)rYsource link_namer4r4r5r,s   r,c@sNeZdZdZ    dddZddZdd Zd d Zdd d ZdddZ dS) state_updateaThis context manager is responsible for dispatching the state-enter and state-leave signals to the watchman service. The enter and leave methods can be invoked manually (for scenarios where context manager semantics are not possible). If parameters oldnode and newnode are None, they will be populated based on current working copy in enter and leave, respectively. Similarly, if the distance is none, it will be calculated based on the oldnode and newnode in the leave method.NFcCs8||_||_||_||_||_||_d|_d|_dSr) unfilteredr nameoldnodenewnodedistancepartial_lock need_leave)rZr r5r6r7r8r9r4r4r5rs  zstate_update.__init__cCs |dSry)enterrr4r4r5 __enter__s zstate_update.__enter__cCsh|jdur |jd|_|jdur(t|jdr"|j|_n|j|_| dt |j|_ |S)Nrswlocknostateupdates state-enter) r6r node currentwlockrrwlocknostateupdater:r_staterr;rr4r4r5r<s  zstate_update.entercCs|rdnd}|j|ddS)NTF)abort)exit)rZtype_valuetbrBr4r4r5__exit__*s zstate_update.__exit__cCsz@|jr1|rdnd}|jdur|jd|_|jdur&t|j|j|j|_|jdt|j|dWd|_|j r?|j dSdSd|_|j rM|j ww)Nsfailedokrs state-leave)rF) r;r7r r>r8 calcdistancer6rArr:release)rZrBrr4r4r5rC.s&     zstate_update.exitrHc Cst|jds dSz|jj||j||jr|jnd||jddWdStyA}z|jj dd||WYd}~dSd}~ww) Ns_watchmanclientFr)srevsdistancesstatusspartial)rksmetadataTrsException %s while running %s ) rrr rr(r5r8r9rr-r<)rZcmd commithashrr3r4r4r5rA>s*zstate_update._state)NNNF)F)rH) rrr__doc__rr=r<rGrCrAr4r4r4r5r3s   r3cCsF|j||}||}t|||t|||}|Sry) changelogancestorrabs)r r6r7ancancrevr8r4r4r5rI\s  rIc  Ksd} d} |d} ||} |dus|r"d} t|| | } t|d| | | | d|||||||||fi| WdS1sFwYdS)NrTrFs hg.update)r5r6r7r8r9)r>rrIr4r3)rYr r> branchmergeforcerO mergeancestorlabelsmatcherrr8r9r6r7r4r4r5r-is8    $r-cCsH|jD]}tjtj|j|dr!d}|j||dSqdS)Nrhs:fsmonitor: sub-repository %r detected, fsmonitor disabled TF) wvfslistdirrrPisdirrr/r-rV)r rcr=r4r4r5repo_has_depth_one_nested_reposr[c st}tD]}||vrtd|dSq|r|jds)|jdr+dSt|r1dSt |}|j dkr=dSz t |j |j}Wnty`}z t||WYd}~dSd}~ww||_||_t|d\}}|rvt||Gfddd|j|_dSdS)NsUThe fsmonitor extension is incompatible with the %s extension and has been disabled. s .hgsubstates.hgsubsoffr'cs<eZdZfddZfddZfddZZS)z reposetup..fsmonitorrepocs$t|j}t||g|Ri|Sry)rrrrr fsmonitorrepor4r5rs z'reposetup..fsmonitorrepo.statuscst|j|i|Sry)rrrr\r4r5r@sz3reposetup..fsmonitorrepo.wlocknostateupdatec st|j|i|ddsSjdkrSjfdd}zd_t|dd_j|_WStyT}z|j dd |WYd}~Sd}~ww) Nr r!rcs(rjrjd_dSdSry) stateupdaterCr4l origreleaser4r5 statereleases   z.fsmonitorrepo.wlock..statereleaseshg.transaction)r5rsException in state update %s ) rrr:held releasefnr^r3r<rr-r<)rZrrrbr3)rr]r-r_r5rs,  z&reposetup..fsmonitorrepo.wlock)rrrrr@rr!r4r]r-r#r5r]sr])r enabled _blacklistr9rlocalrXexistsr[rrrr'r-r/rr?rrr isfilecachedr$r) r-r extsextfsmonitorstater'r>rrr4rer5 reposetupsF      %rnrq)rNNFFFF)NFNN)ArM __future__rcodecsrrsysr%rmercurial.i18nrmercurial.nodermercurial.pycompatr mercurialrrrr r r r r rrrrrmercurial.utilsrrrrr testedwith configtable configitemlistrgr6r?rFget_local_encodingrJgetfilesystemencodinggetdefaultencodingrNlookuprrSrrobjectrr$r&r.r,r3rIr-r[rnr4r4r4r5s f   4     |   ` *