o vfN@sdZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl m Z ddl mZejjdkr[ddlmZmZmZddlmZddlmZmZeeeeeefd Zndd lmZmZmZmZdd lmZddlmZd Zzdd lmZddl m!Z!e!Wn e"ydZ!Ynwddl#Z$ddl$Z$ej%&dZ'ddZ(ddZ)Gddde$j*j+Z+da,GdddeZ-GdddeZ.d!ddZ/e0dkrddl1Z1ddl2Z2ddl3m4Z4da*da5da6da7Gdd d e1j8Z9e1:dSdS)"z,Crash database implementation for Launchpad.N)FailedToDecompressContent)BytesIO) HTTPSHandlerRequest build_opener)HTTPSConnection) urlencodeurlopenT)rrrr )r F) HTTPError) Launchpadz%~/.cache/apport/launchpad.credentialsc cs`|D]*}z|j}WnttfytdYqw|j}|ds*|dr-|VqdS)Nz"Broken attachment on bug, ignoring.txt.gz)dataopenr rapporterrorfilenameendswith) attachments attachmentfnamer?/usr/lib/python3/dist-packages/apport/crashdb_impl/launchpad.pyfilter_filename,s rcCstdd|DS)Ncss$|] }t|jdVqdS)/N)int self_linksplitpop).0irrr :s"zid_set..)set)tasksrrrid_set8sr&c@seZdZdZddZeddZddZedd ZdAd d Z d dZ ddZ ddZ ddZ  dBddZdCddZddZddZdd Zd!d"Zd#d$Zd%d&Zd'd(Zd)d*Zd+d,Zd-d.Zd/d0Zd1d2Zd3d4ZdAd5d6Zd7d8Zd9d:Zd;d<Z d=d>Z!e"d?d@Z#d S)D CrashDatabasez5Launchpad implementation of crash database interface.cCstdr td|d<|s$|d}|r"td|ddd}nt}tjj||||d|_ |j r?d|vs>Jd nd|vsGJd d |vrSd |d |_ nd tj |_ ||_ ||_|jsfJd |_d |_td|d|_|jstjdd|_tjtj|jddd Sd S)aInitialize Launchpad crash database. You need to specify a launchpadlib-style credentials file to access launchpad. If you supply None, it will use default_credentials_path (~/.cache/apport/launchpad.credentials). Recognized options are: - distro: Name of the distribution in Launchpad - project: Name of the project in Launchpad (Note that exactly one of "distro" or "project" must be given.) - launchpad_instance: If set, this uses the given launchpad instance instead of production (optional). This can be overriden or set by $APPORT_LAUNCHPAD_INSTANCE environment. - cache_dir: Path to a permanent cache directory; by default it uses a temporary one. (optional). This can be overridden or set by $APPORT_LAUNCHPAD_CACHE environment. - escalation_subscription: This subscribes the given person or team to a bug once it gets the 10th duplicate. - escalation_tag: This adds the given tag to a bug once it gets more than 10 duplicates. - initial_subscriber: The Launchpad user which gets subscribed to newly filed bugs (default: "apport"). It should be a bot user which the crash-digger instance runs as, as this will get to see all bug details immediately. - triaging_team: The Launchpad user/team which gets subscribed after updating a crash report bug by the retracer (default: "ubuntu-crashes-universe") - architecture: If set, this sets and watches out for needs-*-retrace tags of this architecture. This is useful when being used with apport-retrace and crash-digger to process crash reports of foreign architectures. Defaults to system architecture. APPORT_LAUNCHPAD_INSTANCElaunchpad_instance.z://distroprojectz/Must not set both "project" and "distro" optionz0Need to have either "project" or "distro" option architecturezneed-%s-retraceNAPPORT_LAUNCHPAD_CACHE cache_dirzlaunchpadlib.cache.)prefixT) ignore_errors)osgetenvgetdefault_credentials_pathrrcrashdbr'__init__r-arch_tag packagingget_system_architectureoptionsauth_CrashDatabase__launchpad_CrashDatabase__lp_distro_CrashDatabase__lpcachetempfilemkdtempatexitregistershutilrmtree)selfr>r= lp_instancerrrr9@s6 !   zCrashDatabase.__init__c Cs|jr|jStdurtrtjdntjdtd|jdr+|jd}nd}t j |j }|rAt j |sAt |ztjd||jdg|j d d |_W|jSty}z"t|d re|j}nt|}td ||j td WYd}~|jSd}~ww)zReturn Launchpad instance.Nz^ERROR: The python-launchpadlib package is not installed. This functionality is not available. z_ERROR: The python3-launchpadlib package is not installed. This functionality is not available. r+r) productionzapport-collect WRITE_PRIVATEz1.0)launchpadlib_dirallow_access_levelscredentials_fileversioncontentzZconnecting to Launchpad failed: %s You can reset the credentials by removing the file "%s"c)r?r _python2sysstderrwriteexitr=r6r4pathdirnamer>isdirmakedirs login_withrA ExceptionhasattrrPstrrr)rHr)auth_diremsgrrr launchpads>      zCrashDatabase.launchpadccsH|jst|D]}|j|jkstd|j|jr!|VqdS)Nz ^.+\(%s.*\)$)r- StopIterationbug_target_namelowerrematch)rHr%trrr_get_distro_tasksszCrashDatabase._get_distro_taskscCsX|jdur)|jr|jj|j|_|jSd|jvr%|jj|jd|_|jStd|jS)Nr.z:distro or project needs to be specified in crashdb options)r@r-rb distributionsr=projects SystemErrorrHrrr lp_distros  zCrashDatabase.lp_distroNcCs>||sJ||}t|||d}||sJ|S)aUpload given problem report return a handle for it. This should happen noninteractively. If the implementation supports it, and a function progress_callback is passed, that is called repeatedly with two arguments: the number of bytes already sent, and the total number of bytes to send. This can be used to provide a proper upload progress indication on frontends. )hostname)accepts_generate_upload_blob upload_blob get_hostnameclose)rHreportprogress_callback blob_fileticketrrruploads  zCrashDatabase.uploadcCs0|jd}|r|dkrd}|Sd}|Sd}|S)z/Return the hostname for the Launchpad instance.r)stagingzstaging.launchpad.netz launchpad.dev launchpad.net)r=r6)rHr)rorrrrss zCrashDatabase.get_hostnamecCsi}|d|}|rt|ts|d}||d<|}d|vr'|d}n|jd}|sLd|vrAd||j|d|t|fSd||j|t|fSd|||t|fS) a/Return an URL that should be opened after report has been uploaded and upload() returned handle. Should return None if no URL should be opened (anonymous filing without user comments); in that case this function should do whichever interactive steps it wants to perform.TitleUTF-8z field.title SnapSourcer. SourcePackagez,https://bugs.%s/%s/+source/%s/+filebug/%s?%sz!https://bugs.%s/%s/+filebug/%s?%s) r6standard_title isinstancebytesencodersr=r-r )rHruhandleargstitleror.rrrget_comment_urls*     zCrashDatabase.get_comment_urlcCs dt|S)zReturn URL for a given report ID. The report is passed in case building the URL needs additional information from it, such as the SourcePackage name. Return None if URL is not available or cannot be determined. z https://bugs.launchpad.net/bugs/)r^)rHruidrrr get_id_urls zCrashDatabase.get_id_urlc Cst}|jj|}td|jtj}|s!td|jtjtjB}|s'Jd| d d dd dd }d |vr`d |vrX| d d\}}| d d d | d dd }n| d dd }| t|d |vrz |j|d <Wnty|j|d <Ynwd|vrd|jvrd|d<n'd|jvrd|d<nd|jvrd|d<nd|jvrd|d<n tdt|jd|j|d<d|vr|d|d<|j|d<t|jD]z}tj|j\} } zd|| <Wn tyYqw| dkr||| <z || d|| <Wqt yYqw| dkrHz t!j"|d|| <Wqt#yG} zd t| vr2|$d ||| <WYd!} ~ qd!} ~ wwtd"|j|S)#z>Download the problem report from given ID and return a Report.z(ProblemType:.*)$z^--- \r?$[\r\n]*(.*)z8bug description must contain standard apport format datar+r}s  s  s sUname:rDate ProblemTypez apport-bugBug apport-crashCrashzapport-kernelcrash KernelCrashzapport-packagePackagez(cannot determine ProblemType from tags:  Tagsr| OriginalTitler r)fileobjz Not a gzipNzUnknown attachment type: )%rReportrbbugsrfsearch descriptionSMgrouprreplacerloadr date_createdctimeAttributeErrortags ValueErrorr^joinrrrr4rWsplitextrr\readdecodeUnicodeDecodeErrorgzipGzipFileIOErrorseek) rHrrubmrpart1part2rkeyextr`rrrdownload sz                    zCrashDatabase.downloadFc Cs|jj|}|rt|t|}nd}t} |j| |d| | dt r1t | } nt | } | } t r@| pC| } | sJJt rP| pS| } | rZJ| dksbJ|s|jdd} | dd|vr| ||d7} | |_||jj|}| jddd d }|r|r|jd ||_|n |s|j}|j||d | D]} |j|pd | d| jdd| ddq| dS)ahUpdate the given report ID with all data from report. This creates a text comment with the "short" data (see ProblemReport.write_mime()), and creates attachments for all the bulk/binary data. If change_description is True, and the crash db implementation supports it, the short data will be put into the description instead (like in a new bug). comment will be added to the "short" data. If attachment_comment is given, it will be added to the attachment uploads. If key_filter is a list or set, then only those keys will be added. N) skip_keysr text/plainapport-collectedrTrr}rz --- rPsubjectrFcommentr content_typerris_patch)rbrr$keysrB TemporaryFile write_mimeflushrrRemailmessage_from_filemessage_from_binary_filewalknext__next__ is_multipartget_content_typerappend_filter_tag_namesrlp_save get_payloadrrr newMessage addAttachment get_filenamert)rHrrurchange_descriptionattachment_comment key_filterbugrmimeramsg_iterpartxtextrrrupdateasR            zCrashDatabase.updaterc Cstjj|||||jj|}d|vr7|jD]}|jj dr6|j j |dd|_| |jj|}nq| r|jD]}|jdkrUz|Wq>tyTYq>wq>z||j}trd|}n|}|jdkrtd|_| Wn ty~Ynw|}|rtd|j}|r|d|kr|d ||d |_z| Wn tyYnw|jj|}|||d S) zUpdate the given report ID for retracing results. This updates Stacktrace, ThreadStacktrace, StacktraceTop, and StacktraceSource. You can also supply an additional comment. r #distributionr CoreDump.gz UndecidedMediumz,^(.*crashed with SIG.* in )([^( ]+)(\(\).*$)rr+N)rr8r' update_tracesrbr bug_taskstargetresource_type_linkrrngetSourcePackagerhas_useful_stacktracerr removeFromBugr rirRrr importancercstacktrace_top_functionrfrgr_subscribe_triaging_team) rHrrurrtaskafnrrrrrsV              zCrashDatabase.update_tracescCs0|jj|}td|j}|r|dStd)zNGet 'DistroRelease: ' from the given report ID and return it.z"DistroRelease: ([-a-zA-Z0-9.+/ ]+)r+z)URL does not contain DistroRelease: field)rbrrfrrrr)rHrrrrrrget_distro_releases  z CrashDatabase.get_distro_releasecCs`td|j}|jj|}g}|jD]}||jj}|sq|j dvr%q| | dq|S)z5Return list of affected source packages for given ID.z5/%s/(?:(?P[^/]+)/)?\+source/(?P[^/]+)$)Invalid Won't Fix Fix Releasedsource) rfcompiler-rbrrrrrstatusrr)rHr bug_target_rerresultrrgrrrget_affected_packagess   z#CrashDatabase.get_affected_packagescCs|jj|}|jj|jjjkS)z3Check whether the user is the reporter of given ID.)rbrownerrmerHrrrrr is_reporters zCrashDatabase.is_reportercCsZ|jj|}|jr dS|jj|jjjkrdS|jjj}|jjD] }|d|kr*dSqdS)aACheck whether the user is eligible to update a report. A user should add additional information to an existing ID if (s)he is the reporter or subscribed, the bug is open, not a duplicate, etc. The exact policy and checks should be done according to the particular implementation. FT person_link) rbr duplicate_ofrrrr subscriptionsentries)rHrrrsubrrr can_update s    zCrashDatabase.can_updatec Cs^z|jj|jdd}t|WSty.}ztdt|t dWYd}~dSd}~ww)z}Return an ID set of all crashes which have not been retraced yet and which happened on the current host architecture. 2011-08-01r created_since"connecting to Launchpad failed: %srQN) rn searchTasksr:r&r\rrr^rSrVrHrr`rrrget_unretraced s zCrashDatabase.get_unretracedc Cs\z |jjddd}t|WSty-}ztdt|tdWYd}~dSd}~ww)a#Return an ID set of all crashes which have not been checked for being a duplicate. This is mainly useful for crashes of scripting languages such as Python, since they do not need to be retraced. It should not return bugs that are covered by get_unretraced().need-duplicate-checkrrrrQN) rnrr&r\rrr^rSrVr rrrget_dup_unchecked*s zCrashDatabase.get_dup_uncheckedcCs|jjdd}t|S)aOReturn an ID set of all crashes which are not yet fixed. The list must not contain bugs which were rejected or duplicate. This function should make sure that the returned list is correct. If there are any errors with connecting to the crash database, it should raise an exception (preferably IOError).r)r)rnrr&)rHrrrr get_unfixed9s zCrashDatabase.get_unfixedcCs"|jjjd||jjd}|djS)zReturn the version of given source package in the latest release of given distribution. If 'distro' is None, we will look for a launchpad project . T) exact_match source_name distro_seriesr)rn main_archivegetPublishedSourcescurrent_seriessource_package_version)rHpackagesourcesrrr_get_source_versionEs  z!CrashDatabase._get_source_versionc sNzjj|}Wn tyYdSw|jrdSt|j}jrdjttfdd|}|sCttfdd|}|rCdSt |dkrSt dj|dS|ro| }z  |jd WStynYdSwttfd d|}|rttfd d|}|sdSdSttd d|}|rdSttd d|rdSdS)aReturn the package version that fixes a given crash. Return None if the crash is not yet fixed, or an empty string if the crash is fixed, but it cannot be determined by which version. Return 'invalid' if the crash report got invalidated, such as closed a duplicate or rejected. This function should make sure that the returned result is correct. If there are any errors with connecting to the crash database, it should raise an exception (preferably IOError). invalidz(%s)cs|jdko |jvSNrrbug_target_display_namererdistro_identifierrro z1CrashDatabase.get_fixed_version..cs|jdko|jjkSr)rrdrer-rrmrrrss rr+zVThere is more than one task fixed in %s %s, using first one to determine fixed versionrcs|jdvo |jvSN)rrExpiredrrrrrrr cs|jdvo |jvSr!rrrrrrr cS |jdkSrrrrrrr cSr#)Nrr$rrrrrr%N)rbrKeyErrorrlistrr-refilterlenrwarningr rrr IndexError) rHrrr% fixed_tasks fixed_distror invalid_tasksnon_invalid_tasksr)rrHrget_fixed_versionSs\      zCrashDatabase.get_fixed_versioncCs|jj|j}|r |jSdS)zcReturn master ID for a duplicate bug. If the bug is not a duplicate, return None. N)rbrrr)rHrrrrrrszCrashDatabase.duplicate_ofc Cs|jj|}|rT||ksJdt||jj|}|jr2|j}|j}|j|kr2td||dS|jD]}|jdvrLz| Wq5t yKYq5wq5|jj|}|j d|dd|jj|}|j rhd|_ |jsn||_|j }t|jd krd |jvr|jd |vr|jd d |vr||jd g|_ |d |jvr|jd d |vr|jj|jd }|j|d|rd|vsd|vr|j||d|gdd|jj|}|j dd} z| dWn tyYnwz| dWn tyYnw| |_ z|Wn t yYnwdg} |jjD]} | jdvr'q| | jqt|j } |j }| |} | D] }|| vrJ||q>||_ |n|jr[d|_|j re|dSdS)zlMark a crash id as duplicate of given master ID. If master is None, id gets un-duplicated. z+cannot mark bug %s as a duplicate of itselfzNBug %i was manually marked as a dupe of newer bug %i, not closing as duplicateN)rzStacktrace.txtzThreadStacktrace.txtz ProcMaps.txtzProcStatus.txtz Registers.txtzDisassembly.txtaThank you for taking the time to report this crash and helping to make this software better. This particular crash has already been reported and is a duplicate of bug #%i, so is being marked as such. Please look at the other bug report to see if there is any missing information that you can provide, or to see if there is a workaround for the bug. Additionally, any further discussion regarding the bug should occur in the other report. Please continue to report any other bugs you may find.zThis bug is a duplicaterF escalation_tag escalated_tagz invalid escalation_subscriptionpersonzapport-request-retraceapport-failed-retracez)Updated stack trace from duplicate bug %i) StacktraceThreadStacktracer DependenciesProcMaps ProcCmdline)rzbugpattern-needed)zActive DevelopmentzCurrent Stable Release SupportedzPre-release Freeze)!rbrr^rrrr*rrrr rprivaterr) duplicatesr=r6rpeople subscriberrremoverrnseriesrrrr$ difference_dirty_attributes)rHrur master_idrmasterr master_tagspr tags_to_copyrC dupe_tags missing_tagstagrrrclose_duplicates         *             zCrashDatabase.close_duplicatecCsD|jj|}|jd|dd|jj|}|jdg|_|dS)zpMark a crash id as reintroducing an earlier crash which is already marked as fixed (having ID 'master').zThis crash has the same stack trace characteristics as bug #%i. However, the latter was already fixed in an earlier package version than the one in this report. This might be a regression or because the problem is in a dependent package.zPossible regression detectedrregression-retracerN)rbrrrr)rHrrGrrrrmark_regressions   zCrashDatabase.mark_regressioncCs`|jj|}|j|jvr.|jdd}||j||_z|WdSty-YdSwdS)zMark crash id as retraced.N)rbrr:rrBrr )rHrrrrrr mark_retraced#s    zCrashDatabase.mark_retracedc Cs|jj|}|rUz||j}tr|}n|}Wnty)|jd}Ynwd|_| |j |dd|j D]}|j dkrRz| Wq;tyQYq;wq;dSd|jvrg|jdg|_| dSdS)z%Mark crash id as 'failed to retrace'.rrz Crash report cannot be processedrrr7N)rbrrirrRrrrcrrrrrrr r)rHr invalid_msgrrrrrrmark_retrace_failed0s:          z!CrashDatabase.mark_retrace_failedc Cs|jj|}d|vr9|jD]+}|jjdr8|jj|dd|_z ||jj|}Wn t y5Ynwnq d|j vrm|j dd}| d||_ |d|vrm|jD]}d|jjvrl|j dkrld|_ |qX| ||dS) z/Mark crash id as checked for being a duplicate.rrrr N Tracebackrr)rbrrrrrrnrrr rrBrr)rHrrurrrrrr_mark_dup_checkedNs8        zCrashDatabase._mark_dup_checkedcCstjj||}|s |S||d<zt|d}Wn ty#|YSw|}|ds/dS|D]}|drFd|vs@d|vrCdS|S|sM|Sq1|S) aCheck if the crash db already knows about the crash signature. Check if the report has a DuplicateSignature, crash_signature(), or StacktraceAddressSignature, and ask the database whether the problem is already known. If so, return an URL where the user can check the status or subscribe (if available), or just return True if the report is known but there is no public URL. In that case the report will not be uploaded (i. e. upload() will not be called). Return None if the report does not have any signature or the crash database does not support checking for duplicates on the client side. The default implementation uses a text file format generated by duplicate_db_publish() at an URL specified by the "dupdb_url" option. Subclasses are free to override this with a custom implementation, such as a real database lookup. DuplicateOfz/+textsbug:Tstags:sapport-failed-retracesapport-request-retraceN) rr8r'knownr rreadline startswithstrip)rHruurlrlinerrrrWns.   zCrashDatabase.knowncCsxd|vr|dddkrdSd|jj|jddf}|t|jjdd d d |jDvr:|j |d dSdS) z-Subscribe the right triaging team to the bug. DistroReleaserUbuntuNz%s~%s triaging_teamzubuntu-crashes-universer~cSsg|] }t|ddqS)rr,)r^r)r!rrrr sz:CrashDatabase._subscribe_triaging_team..r5) rrb _root_urir=r6rr^rZrrA)rHrrur6rrrrs z&CrashDatabase._subscribe_triaging_teamcCsni}d|d|d<|d}|r|dkr|d}|r(|dd|7<d|vr;|dd||d7<d|vr}|rid |vsMd |vsMd |vrid |d <|d|jdd|d<|dd|7<nd|vr}d |d <d|d<|dd7<d|vrd|dvr|dd7<d|vr|d|d<gd}t}|j||gd|d||d|S)zGenerate a multipart/MIME temporary file for uploading. You have to close the returned file object after you are done with it. z apport-%srrPackageArchitectureall Architecturerr]VmCoreCoreDumpLaunchpadPrivateyesPrivateLaunchpadSubscribeinitial_subscriberr Subscribersz need-%s-retracerTz need-duplicate-checkDuplicateSignaturer CheckboxSubmissionzHWDB-Submission) rr]r Regression ReproducibleTestedUpstreamProcVersionSignatureUnameNonfreeKernelModules)rrhrk) extra_headersrpriority_fieldsr) rer6rr=rBrrrr)rHruhdrrorderrrrrrqsB      z#CrashDatabase._generate_upload_blobcCs`d}|jdddD]"}|dvst|dkr)|dvr)tr"||7}q |t|7}q |d7}q |S) zAReplace characters from tags which are not palatable to LaunchpadrASCIIignore)errorss%abcdefghijklmnopqrstuvwxyz0123456789 rs+-.r*)rerr)rRchr)klassrreschrrrrs  zCrashDatabase._filter_tag_namesN)FNN)r)$__name__ __module__ __qualname____doc__r9propertyrbrirnryrsrrrrrrrrrr r r rr0rrNrPrQrSrUrWrrq classmethodrrrrrr'=sJB (   # U S4   F q  72r'c@seZdZdZddZdS)HTTPSProgressConnectionzWImplement a HTTPSConnection with an optional callback function for upload progress.cCsts t||dSd}t|}d}||krRt||t}t|||||||7}t}|dkrL||dkrB|dK}n ||dkrL|dL}||ksdSdS)Nrig?r+r)_https_upload_callbackrsendr)time)rHrsenttotal chunksizet1t2rrrrs$     zHTTPSProgressConnection.sendN)rrrrrrrrrrs rc@seZdZddZdS)HTTPSProgressHandlercCs |t|Sr)do_openr)rHreqrrr https_open$s zHTTPSProgressHandler.https_openN)rrrrrrrrr"s rr{c Csd}d|}|atjj}tjjd}|dd||tjj dd}|dd| | d ||t}tjjd krOtjj|d d } ntjj|d d } | |t||} | d d|tt} | | } | d}|sJ|S)a Upload blob (file-like object) to Launchpad. progress_callback can be set to a function(sent, total) which is regularly called with the number of bytes already sent and total number of bytes to send. It is called every 0.5 to 2 seconds (dynamically adapted to upload bandwidth). Return None on error, or the ticket number on success. By default this uses the production Launchpad hostname. Set hostname to 'launchpad.dev' or 'staging.launchpad.net' to use another instance for testing. Nzhttps://%s/+storeblob1zContent-Dispositionzform-data; name="FORM_SUBMIT" applicationz octet-streamz*form-data; name="field.blob"; filename="x"asciirF) mangle_from_z Content-Typezmultipart/form-data; boundary=zX-Launchpad-Blob-Token)rrr multipart MIMEMultipartrMIMEText add_headerattachbaseMIMEBase set_payloadrrrrS version_infomajor generator GeneratorBytesGeneratorflattenrgetvalue get_boundaryrrrinfor6) blobrvrorxr[rsubmit form_blob data_flatgenropenerrrrrrr(s.        rr__main__)patchc@sTeZdZdZdZddZddZeddZdNd d Z d d Z dNd dZ ddZ ddZ ddZddZddZddZddZddZdd Zd!d"Zd#d$Zd%d&Zd'd(Zd)d*Zd+d,Zd-d.Zeed/d0d1Ze d2d3Z!d4d5Z"dOd7d8Z#d9d:Z$d;d<Z%d=d>Z&d?d@Z'dAdBZ(dCdDZ)dEdFZ*dGdHZ+dIdJZ,e dPdLdMZ-d6S)Q_T coreutilscCsHts|at|_t|_|j|jd|jd<|ddS)Nrrlangpack-o-matic)r8 _get_instancerr ref_report add_os_info add_user_info_create_projectrmrrrsetUprs    z_T.setUpcCs@|jjj|}|s|jjjj|d|||d|dddSdS)z8Create a project using launchpadlib to be used by tests.rsummaryr)r display_namerrrN)r8rbrk new_project)rHrr.rrrrs  z_T._create_projectcCs |jS)z1Get the Launchpad hostname for the given crashdb.)r8rsrmrrrros z _T.hostnameFcCs|stdurtS|}||j|||||dd|d<d|d<| |j |}| || ||}| |dkt jd|j|f|sV|a|S) zGenerate SEGV crash report. This is only done once, subsequent calls will return the already existing ID, unless force_fresh is True. Return the ID. N!crash crashed with SIGSEGV in f()u "]¶" ShortGibberisha b c d e ÿÿÿ LongGibberishrz*(Created SEGV report: https://%s/bugs/%i) ) _segv_report_generate_sigsegv_reportadd_package_info test_packager add_gdb_infor assertEqualr_get_bug_targetr8 assertTrue _file_bugrSrTrUro)rH force_freshr bug_targetrrrrget_segv_reports$    z_T.get_segv_reportcCstdurtStd}d|d<d|d<d|d<||j||||d | |j |}| || ||}| |d kt jd |j|f|a|S) zFGenerate Python crash report. Return the ID. Nr/bin/fooExecutablePathTraceback (most recent call last): File "/bin/foo", line 67, in fuzz print(weird) NameError: global name 'weird' is not definedrTzboogus pybogusrHfoo crashed with NameError in fuzz(): global name 'weird' is not definedrz,(Created Python report: https://%s/bugs/%i) )_python_reportrrrrrrrrrr8rrrSrTrUro)rHrrrrrrget_python_reports&     z_T.get_python_reportcCsV|stdurtSd}|jjjjd||jjd}tjd|j |j f|s(|j a|j S)aFile a bug report with an uncommon description. This is only done once, subsequent calls will return the already existing ID, unless force_fresh is True. Example taken from real LP bug 269539. It contains only ProblemType/Architecture/DistroRelease in the description, and has free-form description text after the Apport data. Return the ID. Nzbproblem ProblemType: Package Architecture: amd64 DistroRelease: Ubuntu 8.10 more text and more zmixed description bug)rrrz3(Created uncommon description: https://%s/bugs/%i) ) _uncommon_description_reportr8rbr createBugrnrSrTrUror)rHrdescrrrrget_uncommon_description_reports  z"_T.get_uncommon_description_reportc Cs|j|}||dd||dd||d|jd||d|jd||d|jd||d|jd||d |jd t|d }||t|jjd t j g||d d | |d d||d|j| |d|jd|d|d|d|d|d|d|t|dd|d||d||d||j|}t|d }||td dddt j gdS) z download()rrr|rr]rertru UserGroupsrrSignal11rz/crashrrrzf (x=42)r8 StacktraceTopr9rgir: Disassembly Registersbooguspybogusr N)r8rrrrr6r$rr:rr;r<rrtest_srcpackagerYrassertIn assertGreaterr)r)rHrrrrrtest_1_downloads<       z_T.test_1_downloadcCs|j|}|d||d||d||d||d||d|||ddd |d <d |d<d |d<d |d<|j||d|j|}|d||d||d||d||d||d||d|||dd|jjj|j }|d||d|d|d <d |d<d |d<|j||d|j|}|d||d||d||d||d||d||d|||dd|jjj|}d|_ z| Wn t y Ynwd|d <|j||d|j|}||dd|jjj|}d|_ z| Wn t yEYnwd|d <|j||d|j|}||ddd|d<d|d<d|d<|j||ddS) zupdate_traces()rgr:rrr8r9r|rz?? ()r long tracethread even longer tracebogusFooBarzI can has a better retrace?rr=read () from /lib/libc.6.so foo (i=1) from /usr/lib/libfoo.so good retrace!z$crash crashed with SIGSEGV in read()z)crash crashed with SIGSEGV in f() on exitz!good retrace with title amendmentz,crash crashed with SIGSEGV in read() on exitzcrash is crashyzgood retrace with custom titleru"]¶" rStacktraceSourcetestsN) r8rrrrr assertNotInrbrrrrr )rHrrrrrrtest_2_update_tracess                        z_T.test_2_update_tracescCsd}td}||d<|j|d}||d|d|d<|j|d}||dd|d<|j|d}||d d S) z&get_comment_url() for non-ASCII titless1ä♥2rr|*z1/ubuntu/+filebug/42?field.title=1%C3%A4%E2%99%A52r}rrzC/ubuntu/+source/coreutils/+filebug/42?field.title=1%C3%A4%E2%99%A52N)rrr8rrrr)rHrrr[rrrtest_get_comment_urlms z_T.test_get_comment_urlcCs |jjjdd}|jjjjd|dd}|j}||dktj d|j |ft d}d d |d <d |d <d|d<d|d<d|d<|jj||ddd|j|}||d d d ||dd||dd||dd||jjj|jdgdS)z"update() with changing descriptionbashrztest description for test bug.testbugrrrr(https://%s/bugs/%i) rsbogus→r}OneLiner f() g() h(1)rlineone linetwoShortGooone two three four five sixDpkgTerminalLogsVarLogDistupgradeBinGooNotMeTrrN)r8rnrrbrrrrrSrTrUrorrrrrrrrHrrrrrrrtest_update_descriptions0   z_T.test_update_descriptioncCs|jjjdd}|jjjjd|dd}|j}||dktj d|j |ft d}d |d <d |d <d |d<d|d<d|d<|jj||ddd|j|}|d ||d|||dd||dd||dd||jjj|jdgdS)zupdate() with appending commentrrzPr0blem --- ProblemType: Bugrrrrrbogus→rrrrrrrrmeowFrrrNr8rnrrbrrrrrSrTrUrorrrrrrrrrrrtest_update_comments2     z_T.test_update_commentcCs |jjjdd}|jjjjd|dd}|j}||dktj d|j |ft d}d |d <d |d <d |d<d|d<d|d<|jj||ddgdd|j|}|d |||dd ||dd||dd|d|||jjj|jgdS)zupdate() with a key filterrrztest description for test bugrrrrrrrrrrrrrrrrT)rrr)rrrNrrrrrtest_update_filters2     z_T.test_update_filtercCs"||j||jddS)zget_distro_release()r]N)rr8rrrrmrrrtest_get_distro_releasesz_T.test_get_distro_releasecCs$||j||jdgdS)zget_affected_packages()rN)rr8rrrrmrrrtest_get_affected_packagess z_T.test_get_affected_packagescC,||j|||jddS)z is_reporter()r+N)rr8rr assertFalsermrrrtest_is_reporterz_T.test_is_reportercCr)z can_update()r+N)rr8rrr rmrrrtest_can_updater z_T.test_can_updatecCs||j|d||j|d|}|}|jdd}|j|}|j|||||j|||j|||||j||||j|d|j||d||j|d||j|d|j|}|d||d||d||d||d ||d ||jt |||j|||||j|||jt |d|jt |d|j||d|jt |d||j|d|j ||| |dS) zduplicate handlingNTrrrgrr; ProcStatusrr9) rr8rrr0rrrNrrrrP_verify_marked_regression)rHsegv_id known_test_idknown_test_id2rrrrtest_duplicatessF        z_T.test_duplicatescCs|j}|||||||j||j}||||||t |g||j |d| ||j||j ||j}||||||t |g||j |d| ||j||j |d|j}||||||t |g||j |ddS)z-processing status markings for signal crashesNzI don't like your) r8r rrrrrQrunionr$r0_mark_needs_retracerS)rHunretraced_beforeunretraced_afterrrrtest_marking_segv5s<    z_T.test_marking_segvcCs|jjjjd|jj|jjdd}tjdpd}t tjdd|d}|jjjd |j|jd d}| }| |j || |j |||j | }| |j ||||t|j g||j|j d d S) 0processing status markings for a project CrashDBfoozubuntu distro retrace bugrrrrr(rzLP_CREDENTIALSrr.r)barzproject retrace bugN)r8rbrrr:rnr4environr6r'r rrrrQrrr$r0)rH distro_bugr) project_db project_bugrrrrrtest_marking_projectXs6   z_T.test_marking_projectcCstjdpd}ttjdd|dd}|}|j}|jjjjddg|jj d d }t d |j |}|j}| ||| || t|j gd S) rr(rzrubuntufakearch)r-r)r/rzneed-fakearch-retracez&ubuntu distro retrace bug for fakearchrz4fake arch bug: https://staging.launchpad.net/bugs/%iN)r4rr6r'r r8rbrrrnprintrrrr$)rHr) fakearch_dbfakearch_unretraced_beforesystemarch_unretraced_beforerfakearch_unretraced_aftersystemarch_unretraced_afterrrrtest_marking_foreign_arch{s,     z_T.test_marking_foreign_archcCs|j}|||||||j||j|j}|||||| t |g||j |ddS)z2processing status markings for interpreter crashesN) r8r rrrrrUrrrr$r0)rHunchecked_beforeunchecked_afterrrrtest_marking_pythons  z_T.test_marking_pythoncCsp|jdd}|j|}|j|||d|d<d|d<d|d<|j||d |j|}|d |d S) zupdating an invalid crash This simulates a race condition where a crash being processed gets invalidated by marking it as a duplicate. Tr rrrr8rr9rrgN)rr8rrNrr)rHrrrrrtest_update_traces_invalids   z_T.test_update_traces_invalidrcGs\dtj_|||j|}||d||||j|ddS)zget_fixed_version() for fixed bugs Other cases are already checked in test_marking_segv() (invalid bugs) and test_duplicates (duplicate bugs) for efficiency. z3.14N) r'r return_value_mark_report_fixedrr8r0r_mark_report_new)rHr fixed_verrrrtest_get_fixed_versions   z_T.test_get_fixed_versioncCs(tjdpd}ttjdd|dS)zCreate a CrashDB instancer(rzrr$)r-r))r4rr6r')r~r)rrrrs  z_T._get_instancecCs<|jd}d|vr|jj|ddS|r|jj|S|jS)z&Return the bug_target for this report.r.rr)r=r6rnrrbrk)rHdbrur.rrrrs  z_T._get_bug_targetNc CsL|durd}|j|}trt|}nt|}||}tr'|p*| }| s1Jtr7|p:| }| rAJ| dksIJ|d|j dd dd7}|jjjj||d ||d |d } |D]}| rwJ| jd |d|j dd|d dqo|dD]} |jjj| } | r| j| dq| jS)zLFile a bug report for a report. Return the bug ID. Nzsome descriptionrz Trr}rrr|rrFrrmr5)r8rqrRrrrrtrrrrrrrrbrrrr6rrrr@rAr) rHrrurrrarheaderrr subscriberrrrrrsD           z _T._file_bugcCs>|jjj|}|jj|jvr|j|jjg|_|dSdS)z$Mark a report ID as needing retrace.N)r8rbrr:rrrrrrrs  z_T._mark_needs_retracecCs6|jjj|}d|jvr|jdg|_|dSdS)z,Mark a report ID as needing duplicate check.r N)r8rbrrrrrrr_mark_needs_dupcheck$s   z_T._mark_needs_dupcheckcCB|jjj|}t|j}t|dksJ|d}d|_|dS)zClose a report ID as "fixed".r+rrNr8rbrr'rr)rrrHrrr%rhrrrr2,   z_T._mark_report_fixedcCr:)zReopen a report ID as "new".r+rNewNr;r<rrrr36r=z_T._mark_report_newcCs |jjj|}|d|jdS)z.Verify that report ID is marked as regression.rON)r8rbrrrrrrrr@sz_T._verify_marked_regressioncCsvtjdpd}ttjdd|d}||jdtd}d|d <d |d <|| || d | ||}||j d| ||}||d ktjd|j|f||}d|d<d|d<d|d<|||d||}|||d|||||||||||d|||d|||d|||ddS)z7reporting crashes against a project instead of a distror(rzrrrNrrrrrTrrrrrrr8rr9rr)r4rr6r'rr-rrrrrrrrrrSrTrUrorrr0rNrr)rHr)r8rrrrrr test_projectFsB       z_T.test_projectcCsH|j|}||dd||dd||dddS)z*download() of uncommon description formatsrrreamd64r]zUbuntu N)r8rrrrrY)rHrrrrtest_download_robustnessvsz_T.test_download_robustnessc sxtjdpd}ttjdd|ddd}d}|jj|jd jd }zst||d D]N}|d 7}t j d || t |||jj|}|jd|jv}tfdd|jD}|dkrp||||q,||||q,Wt|||D]}t j d|| t |dqnt|||D]}t j d|| t |dqwt j ddS)z,Escalating bugs with more than 10 duplicatesr(rzrr$ omgkittenszapport-hackers)r-r)r2r4rr4iV' r+z%i r2csg|]}|jkqSr)r)r!srIrrrasz&_T.test_escalation..r1zR%i N )r4rr6r'rbr@r=rrangerSrTrUrNrrrrranyrr r)rHr)r6count first_duprhas_escalation_taghas_escalation_subscriptionrrErtest_escalationsB     z_T.test_escalationcCs|||j}||||jjj|}|jr'd|_||j d}|jjj d|_ ||j |jjj dd|j|}|j|||j}||||||t|g|jjj|}||j djd||j djd||j djd||j djd ||j djd||j djd ||j|d d S) z9source package task fixup for marking interpreter crashesFrr$r)rr>rr+zcoreutils (Ubuntu)rN)r9rr8r rrbrr>rrrjraddTaskrkrrUrrrr$rdrrr0)rHr-rrhrr.rrrtest_marking_python_task_mangles4   z"_T.test_marking_python_task_manglerc Cs0d}t}tj}zt}tt j |t |t dd }| dWdn1s1wYtgddksAJtjdsIJtjgdtjd tjd s[Jtd gtjgd tjd dksoJtj|d|d <tj|d f|d<||d<|Wt ||St |w)aCreate a test executable which will die with a SIGSEGV, generate a core dump for it, create a problem report with those two arguments (ExecutablePath and CoreDump) and call add_gdb_info(). Return the apport.report.Report. Ncrash.cwzS int f(x) { int* p = 0; *p = x; return x+1; } int main() { return f(42); } )gccz-grPz-ocrashrrS)gdbz--batch--exrunrUzgenerate-core-file corez./crash)stdoutcoresync)readelfz-nrXrrgr)r4getcwdrrurrBrCrDrErFrGchdirrrU subprocesscallrWexistsPIPE check_callrr)r~signalworkdirorig_cwdprfdrrrrs:          z_T._generate_sigsegv_report)Fr)r).rrrrrrrrrorrrrrrrrrrrr r rrr#r,r/r0robjectr'r5rrrrrr9r2r3rr?rArMrOrrrrrrhsV  # $"Q #!6##    :  0 ")r)Nr{);rrBrDos.pathr4rfrrSrrrFhttplib2riorrrurllib2rrrhttplibrurllibr r rRurllib.request urllib.parse http.clientlaunchpadlib.errorsr launchpadlib.launchpadr ImportErrorapport.crashdbrrW expanduserr7rr&r8r'rrrrrrunittestr] unittest.mockrrrrTestCasermainrrrrs|H           F 7