From cheusov@tut.by Fri Dec 14 07:50:49 2007
Return-Path: <cheusov@tut.by>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
by narn.NetBSD.org (Postfix) with ESMTP id 3917D63B8FF
for <gnats-bugs@gnats.netbsd.org>; Fri, 14 Dec 2007 07:50:49 +0000 (UTC)
Message-Id: <s93ir3167g8.fsf@chen.chizhovka.net>
Date: Fri, 14 Dec 2007 09:50:31 +0200
From: cheusov@tut.by
Reply-To:
To: gnats-bugs@NetBSD.org
Subject: more modules for pkgtools/pbulk
X-Send-Pr-Version: 3.95
>Number: 37539
>Category: pkg
>Synopsis: more modules for pkgtools/pbulk
>Confidential:no>Severity: serious
>Priority: low
>Responsible: joerg
>State: closed
>Class: change-request
>Submitter-Id: net
>Arrival-Date: Fri Dec 14 07:55:00 +0000 2007
>Closed-Date: Sat Jan 26 01:58:46 +0000 2008
>Last-Modified: Sat Jan 26 01:58:46 +0000 2008
>Originator: cheusov@tut.by
>Release: NetBSD 4.0_RC5
>Organization:
>Environment:
System: NetBSD chen.chizhovka.net 4.0_RC5 NetBSD 4.0_RC5 (GENERIC) #1: Fri Dec 7 21:33:13 EET 2007 cheusov@chen.chizhovka.net:/srv/obj/sys/arch/i386/compile/GENERIC i386
Architecture: i386
Machine: i386
>Description:
I suggest to separate two modules for pbulk
and allow users to set alternative ones in pbulk.conf.
One for generating pkg_summary.gz and another - for generating ckeck summs.
Patch follows.
>How-To-Repeat:
>Fix:
? summary_cksum.patch
Index: Makefile
===================================================================
RCS file: /cvsroot/pkgsrc/pkgtools/pbulk/Makefile,v
retrieving revision 1.21
diff -u -r1.21 Makefile
--- Makefile 8 Nov 2007 19:39:42 -0000 1.21
+++ Makefile 5 Dec 2007 21:16:44 -0000
@@ -29,7 +29,7 @@
scripts/report scripts/scan scripts/scan-client-start scripts/upload \
scripts/compute-packages.awk scripts/create-broken-graph.awk \
scripts/create-report-html.awk scripts/create-report-txt.awk \
- scripts/create-report.awk
+ scripts/create-report.awk scripts/summary scripts/cksum
SUBST_VARS.tools= AWK BZIP2 DIGEST GZIP_CMD IDENT MAKE MAIL_CMD \
PBULK_CONFIG PKG_INFO PREFIX SH TAR
Index: PLIST
===================================================================
RCS file: /cvsroot/pkgsrc/pkgtools/pbulk/PLIST,v
retrieving revision 1.1.1.1
diff -u -r1.1.1.1 PLIST
--- PLIST 19 Jun 2007 19:49:55 -0000 1.1.1.1
+++ PLIST 5 Dec 2007 21:16:44 -0000
@@ -5,6 +5,7 @@
bin/pbulk-scan
libexec/pbulk/build
libexec/pbulk/build-client-start
+libexec/pbulk/cksum
libexec/pbulk/client-clean
libexec/pbulk/compute-packages
libexec/pbulk/create-broken-graph
@@ -17,6 +18,7 @@
libexec/pbulk/report
libexec/pbulk/scan
libexec/pbulk/scan-client-start
+libexec/pbulk/summary
libexec/pbulk/upload
man/cat1/pbulk-build.0
man/cat1/pbulk-resolve.0
Index: files/pbulk/pbulk.conf
===================================================================
RCS file: /cvsroot/pkgsrc/pkgtools/pbulk/files/pbulk/pbulk.conf,v
retrieving revision 1.9
diff -u -r1.9 pbulk.conf
--- files/pbulk/pbulk.conf 17 Oct 2007 20:42:30 -0000 1.9
+++ files/pbulk/pbulk.conf 5 Dec 2007 21:16:44 -0000
@@ -98,6 +98,8 @@
script_phase_pre_build=@PREFIX@/libexec/pbulk/pre-build
script_phase_build=@PREFIX@/libexec/pbulk/build
+script_phase_summary=@PREFIX@/libexec/pbulk/summary
+script_phase_cksum=@PREFIX@/libexec/pbulk/cksum
script_phase_report=@PREFIX@/libexec/pbulk/report
script_phase_scan=@PREFIX@/libexec/pbulk/scan
script_phase_upload=@PREFIX@/libexec/pbulk/upload
Index: files/pbulk/scripts/Makefile
===================================================================
RCS file: /cvsroot/pkgsrc/pkgtools/pbulk/files/pbulk/scripts/Makefile,v
retrieving revision 1.1.1.1
diff -u -r1.1.1.1 Makefile
--- files/pbulk/scripts/Makefile 19 Jun 2007 19:49:59 -0000 1.1.1.1
+++ files/pbulk/scripts/Makefile 5 Dec 2007 21:16:44 -0000
@@ -2,7 +2,7 @@
SCRIPTS= build build-client-start bulkbuild client-clean \
pkg-build pkg-up-to-date pre-build report scan \
- scan-client-start upload \
+ scan-client-start cksum summary upload \
compute-packages.awk create-broken-graph.awk \
create-report-html.awk create-report-txt.awk \
create-report.awk
Index: files/pbulk/scripts/build
===================================================================
RCS file: /cvsroot/pkgsrc/pkgtools/pbulk/files/pbulk/scripts/build,v
retrieving revision 1.2
diff -u -r1.2 build
--- files/pbulk/scripts/build 20 Jul 2007 19:39:34 -0000 1.2
+++ files/pbulk/scripts/build 5 Dec 2007 21:16:44 -0000
@@ -49,16 +49,3 @@
esac
date '+BUILD_END_ISO=%Y-%m-%d %H:%M' >> ${loc}/status
-
-echo "Building pkg_summary..."
-cd ${packages}/All
-sed 's/$/.tgz/' < ${loc}/success | sort | xargs ${external_pkg_info} -X | ${gzip} -c > pkg_summary.gz
-${gzip} -dc < pkg_summary.gz | ${bzip2} -c > pkg_summary.bz2
-
-echo "Building SHA512..."
-cd ${packages}
-{
- echo "All/pkg_summary.bz2"
- echo "All/pkg_summary.gz"
- sed 's|^\(.*\)$|All/\1.tgz|' < ${loc}/success
-} | sort | xargs ${digest} SHA512 | ${bzip2} -c > SHA512.bz2
Index: files/pbulk/scripts/bulkbuild
===================================================================
RCS file: /cvsroot/pkgsrc/pkgtools/pbulk/files/pbulk/scripts/bulkbuild,v
retrieving revision 1.1.1.1
diff -u -r1.1.1.1 bulkbuild
--- files/pbulk/scripts/bulkbuild 19 Jun 2007 19:49:59 -0000 1.1.1.1
+++ files/pbulk/scripts/bulkbuild 5 Dec 2007 21:16:44 -0000
@@ -8,5 +8,7 @@
${script_phase_pre_build}
${script_phase_scan}
${script_phase_build}
+${script_phase_summary}
+${script_phase_cksum}
${script_phase_report}
${script_phase_upload}
Index: files/pbulk/scripts/cksum
===================================================================
RCS file: files/pbulk/scripts/cksum
diff -N files/pbulk/scripts/cksum
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ files/pbulk/scripts/cksum 5 Dec 2007 21:16:44 -0000
@@ -0,0 +1,13 @@
+#!@SH@
+
+. @PBULK_CONFIG@
+
+set -e
+
+echo "Building SHA512..."
+cd ${packages}
+{
+ echo "All/pkg_summary.bz2"
+ echo "All/pkg_summary.gz"
+ sed 's|^\(.*\)$|All/\1.tgz|' < ${loc}/success
+} | sort | xargs ${digest} SHA512 | ${bzip2} -c > SHA512.bz2
Index: files/pbulk/scripts/summary
===================================================================
RCS file: files/pbulk/scripts/summary
diff -N files/pbulk/scripts/summary
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ files/pbulk/scripts/summary 5 Dec 2007 21:16:44 -0000
@@ -0,0 +1,10 @@
+#!@SH@
+
+. @PBULK_CONFIG@
+
+set -e
+
+echo "Building pkg_summary..."
+cd ${packages}/All
+sed 's/$/.tgz/' < ${loc}/success | sort | xargs ${external_pkg_info} -X | ${gzip} -c > pkg_summary.gz
+${gzip} -dc < pkg_summary.gz | ${bzip2} -c > pkg_summary.bz2
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: pkg-manager->joerg
Responsible-Changed-By: obache@netbsd.org
Responsible-Changed-When: Fri, 14 Dec 2007 11:37:43 +0000
Responsible-Changed-Why:
Over to maintainer.
From: Aleksey Cheusov <cheusov@tut.by>
To: gnats-bugs@NetBSD.org
Cc: joerg@NetBSD.org, pkg-manager@netbsd.org, pkgsrc-bugs@netbsd.org,
gnats-admin@netbsd.org, obache@netbsd.org
Subject: Re: pkg/37539 (more modules for pkgtools/pbulk)
Date: Thu, 27 Dec 2007 23:53:28 +0200
Joerg, I hope you are not against these two submodules. I personally
always disable "cksum" stage and often want to disable "summary" stage.
If you don't want to see them at top level (in bulkbuild script), I
think they can be called from inside "$script_phase_build"
but I think this is worse.
--
Best regards, Aleksey Cheusov.
From: Aleksey Cheusov <cheusov@tut.by>
To: gnats-bugs@NetBSD.org
Cc: joerg@NetBSD.org, gnats-admin@netbsd.org, pkgsrc-bugs@netbsd.org
Subject: Re: pkg/37539 (more modules for pkgtools/pbulk)
Date: Mon, 07 Jan 2008 21:38:09 +0200
> Joerg, I hope you are not against these two submodules. I personally
> always disable "cksum" stage and often want to disable "summary" stage.
> If you don't want to see them at top level (in bulkbuild script), I
> think they can be called from inside "$script_phase_build"
> but I think this is worse.
A bit more explanations. I use pbulk even if I want to build one
partucular package (to test some patches, for example). This is why
want to switch off generating of pkg-summary.gz. When I generate it, I
prefer to use use different, much more efficient, algorithm.
--
Best regards, Aleksey Cheusov.
From: Joerg Sonnenberger <joerg@britannica.bec.de>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/37539 (more modules for pkgtools/pbulk)
Date: Thu, 24 Jan 2008 14:18:10 +0100
On Mon, Jan 07, 2008 at 07:40:03PM +0000, Aleksey Cheusov wrote:
> A bit more explanations. I use pbulk even if I want to build one
> partucular package (to test some patches, for example). This is why
> want to switch off generating of pkg-summary.gz. When I generate it, I
> prefer to use use different, much more efficient, algorithm.
The option to disable checksumming is included.
I don't want to include an option for pkg-summary as I don't see the
need for that. It will create more problems in the future when tools
like pkg_add will depend on it. It is also not that slow. For a full set
of binaries, the generations needs around 3min of time when reading the
packages from disk, 40s of that is actual computation time. Compared to
the build time of even the simple packages, this doesn't fall into the
weight.
Joerg
From: Aleksey Cheusov <cheusov@tut.by>
To: gnats-bugs@NetBSD.org
Cc: joerg@NetBSD.org, gnats-admin@netbsd.org, pkgsrc-bugs@netbsd.org
Subject: Re: pkg/37539 (more modules for pkgtools/pbulk)
Date: Thu, 24 Jan 2008 16:22:50 +0200
>> A bit more explanations. I use pbulk even if I want to build one
>> partucular package (to test some patches, for example). This is why
>> want to switch off generating of pkg-summary.gz. When I generate it, I
>> prefer to use use different, much more efficient, algorithm.
>
> The option to disable checksumming is included.
> I don't want to include an option for pkg-summary as I don't see the
> need for that. It will create more problems in the future when tools
> like pkg_add will depend on it. It is also not that slow. For a full set
> of binaries, the generations needs around 3min of time when reading the
> packages from disk, 40s of that is actual computation time. Compared to
> the build time of even the simple packages, this doesn't fall into the
> weight.
Joerg, this is YOUR future. I already explained you my needs several
times. I don't work with future versions of pkg_add. I work RIGHT NOW
with pkg_add that exists NOW and I manage pkg_summary files in a way I
need. Your pbulk just doesn't allow me to do what I want - managing
multiple binary repositories (like Debian's apt does). The way pbulk
builds summary file is the extra undeed mega-intellect. I don't need
it. My own tool that I use NOW is better than your FUTURE pkg_add just
because it exists NOW. Do you still don't see needs for things I ask
for?
--
Best regards, Aleksey Cheusov.
State-Changed-From-To: open->closed
State-Changed-By: joerg@narn.netbsd.org
State-Changed-When: Sat, 26 Jan 2008 01:58:46 +0000
State-Changed-Why:
checksumming is optional, pkg_summary will not be optional.
>Unformatted:
Home