[Openvas-discuss] Fresh install and problem with openvas-scapdata-sync

Discussion:

Michal Chrobak

2017-01-26 14:16:18 UTC

Hi,

Today I'm trying to install openvas. I downloaded Ubuntu Core 16.04 LTS and install it on KVM virtual machine. Then I add new repository (ppa:mrazavi/openvas from https://launchpad.net/~mrazavi/+archive/ubuntu/openvas) and install openvas 8. Then I run openvas-nvt-sync and openvas-scapdata-sync. And here is my problem, after executing openvas-scap-sync, I've got following error:

***@openvas:~$ sudo openvas-scapdata-sync
[i] This script synchronizes a SCAP data directory with the OpenVAS one.
[i] This script is for the SQLite3 backend.
[i] SCAP dir: /var/lib/openvas/scap-data
[i] Will use rsync
[i] Using rsync: /usr/bin/rsync
[i] Configured SCAP data rsync feed: rsync://feed.openvas.org:/scap-data
OpenVAS community feed server - http://www.openvas.org/
(...)
*rsync cut*
(...)

sent 1,236 bytes received 748,765,593 bytes 815,206.13 bytes/sec
total size is 748,578,705 speedup is 1.00
[i] Initializing scap database
[i] Updating CPEs
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2002.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2003.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2004.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2005.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2006.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2007.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2008.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2009.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2010.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2011.xml
Killed
-:515359: parser error : expected '>'
-:515359: parser error : Premature end of data in tag vulnerable-software-list line 513948
-:515359: parser error : Premature end of data in tag entry line 512501
-:515359: parser error : Premature end of data in tag nvd line 2
unable to parse -
[e] Update of CVEs failed at file '/var/lib/openvas/scap-data/nvdcve-2.0-2011.xml': xsltproc exited with code 137

When I rerun command, it looks that everything is ok, but I'm not convinced that it is true:
***@openvas:~$ sudo openvas-nvt-sync
[i] This script synchronizes an NVT collection with the 'OpenVAS NVT Feed'.
[i] The 'OpenVAS NVT Feed' is provided by 'The OpenVAS Project'.
[i] Online information about this feed: 'http://www.openvas.org/openvas-nvt-feed.html'.
[i] NVT dir: /var/lib/openvas/plugins
OpenVAS community feed server - http://www.openvas.org/
This service is hosted by Greenbone Networks - http://www.greenbone.net/

All transactions are logged.

If you have any questions, please use the OpenVAS mailing lists
or the OpenVAS IRC chat. See http://www.openvas.org/ for details.

By using this service you agree to our terms and conditions.

Only one sync per time, otherwise the source ip will be blocked.

[i] Feed is already current, no synchronization necessary.
***@openvas:~$

My scap-data looks like that:
***@openvas:~$ ls /var/lib/openvas/scap-data/nvdcve-2.0-20*xml -lh
-rw-r--r-- 1 root root 19M Jan 20 09:28 /var/lib/openvas/scap-data/nvdcve-2.0-2002.xml
-rw-r--r-- 1 root root 5.5M Jan 3 09:25 /var/lib/openvas/scap-data/nvdcve-2.0-2003.xml
-rw-r--r-- 1 root root 12M Jan 20 09:26 /var/lib/openvas/scap-data/nvdcve-2.0-2004.xml
-rw-r--r-- 1 root root 18M Jan 20 09:25 /var/lib/openvas/scap-data/nvdcve-2.0-2005.xml
-rw-r--r-- 1 root root 27M Jan 20 09:22 /var/lib/openvas/scap-data/nvdcve-2.0-2006.xml
-rw-r--r-- 1 root root 25M Jan 20 09:20 /var/lib/openvas/scap-data/nvdcve-2.0-2007.xml
-rw-r--r-- 1 root root 31M Jan 20 09:17 /var/lib/openvas/scap-data/nvdcve-2.0-2008.xml
-rw-r--r-- 1 root root 31M Jan 20 09:14 /var/lib/openvas/scap-data/nvdcve-2.0-2009.xml
-rw-r--r-- 1 root root 45M Jan 18 09:09 /var/lib/openvas/scap-data/nvdcve-2.0-2010.xml
-rw-r--r-- 1 root root 109M Jan 7 09:14 /var/lib/openvas/scap-data/nvdcve-2.0-2011.xml
-rw-r--r-- 1 root root 42M Jan 20 09:12 /var/lib/openvas/scap-data/nvdcve-2.0-2012.xml
-rw-r--r-- 1 root root 44M Jan 25 09:58 /var/lib/openvas/scap-data/nvdcve-2.0-2013.xml
-rw-r--r-- 1 root root 40M Jan 25 09:07 /var/lib/openvas/scap-data/nvdcve-2.0-2014.xml
-rw-r--r-- 1 root root 30M Jan 25 09:04 /var/lib/openvas/scap-data/nvdcve-2.0-2015.xml
-rw-r--r-- 1 root root 30M Jan 25 09:02 /var/lib/openvas/scap-data/nvdcve-2.0-2016.xml
-rw-r--r-- 1 root root 442K Jan 25 09:00 /var/lib/openvas/scap-data/nvdcve-2.0-2017.xml

My question is: what go wrong with sync scap-data and how can I correct it?

Regards,
Michal Chrobak

Michal Chrobak
IT Security Systems Engineer
tel. +48 22 122 09 42
tel. +48 503 555 769

SANSEC Poland S.A. NIP: 7010352299, KRS: 0000429238, REGON: 146270315, Spółka zarejestrowana przez Sąd Rejonowy dla M. St. Warszawy w Warszawie, XII Wydział Gospodarczy Krajowego Rejestru Sądowego, Kapitał zakładowy: 1 000 000 PLN.
Niniejsza wiadomość zawiera informacje zastrzeżone i stanowiące tajemnicę przedsiębiorstwa SANSEC Poland S.A.

Eero Volotinen

2017-01-26 14:36:51 UTC

Permalink

you need to add more ram memory to machine.

Eero

Post by Michal Chrobak
Hi,
Today I'm trying to install openvas. I downloaded Ubuntu Core 16.04 LTS
and install it on KVM virtual machine. Then I add new repository
(ppa:mrazavi/openvas from https://launchpad.net/~mrazavi/+archive/ubuntu/
openvas) and install openvas 8. Then I run openvas-nvt-sync and
openvas-scapdata-sync. And here is my problem, after executing
[i] This script synchronizes a SCAP data directory with the OpenVAS one.
[i] This script is for the SQLite3 backend.
[i] SCAP dir: /var/lib/openvas/scap-data
[i] Will use rsync
[i] Using rsync: /usr/bin/rsync
[i] Configured SCAP data rsync feed: rsync://feed.openvas.org:/scap-data
OpenVAS community feed server - http://www.openvas.org/
(...)
*rsync cut*
(...)
sent 1,236 bytes received 748,765,593 bytes 815,206.13 bytes/sec
total size is 748,578,705 speedup is 1.00
[i] Initializing scap database
[i] Updating CPEs
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2002.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2003.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2004.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2005.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2006.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2007.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2008.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2009.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2010.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2011.xml
Killed
-:515359: parser error : expected '>'
-:515359: parser error : Premature end of data in tag
vulnerable-software-list line 513948
-:515359: parser error : Premature end of data in tag entry line 512501
-:515359: parser error : Premature end of data in tag nvd line 2
unable to parse -
xsltproc exited with code 137
When I rerun command, it looks that everything is ok, but I'm not
[i] This script synchronizes an NVT collection with the 'OpenVAS NVT Feed'.
[i] The 'OpenVAS NVT Feed' is provided by 'The OpenVAS Project'.
[i] Online information about this feed: 'http://www.openvas.org/
openvas-nvt-feed.html'.
[i] NVT dir: /var/lib/openvas/plugins
OpenVAS community feed server - http://www.openvas.org/
This service is hosted by Greenbone Networks - http://www.greenbone.net/
All transactions are logged.
If you have any questions, please use the OpenVAS mailing lists
or the OpenVAS IRC chat. See http://www.openvas.org/ for details.
By using this service you agree to our terms and conditions.
Only one sync per time, otherwise the source ip will be blocked.
[i] Feed is already current, no synchronization necessary.
-rw-r--r-- 1 root root 19M Jan 20 09:28 /var/lib/openvas/scap-data/
nvdcve-2.0-2002.xml
-rw-r--r-- 1 root root 5.5M Jan 3 09:25 /var/lib/openvas/scap-data/
nvdcve-2.0-2003.xml
-rw-r--r-- 1 root root 12M Jan 20 09:26 /var/lib/openvas/scap-data/
nvdcve-2.0-2004.xml
-rw-r--r-- 1 root root 18M Jan 20 09:25 /var/lib/openvas/scap-data/
nvdcve-2.0-2005.xml
-rw-r--r-- 1 root root 27M Jan 20 09:22 /var/lib/openvas/scap-data/
nvdcve-2.0-2006.xml
-rw-r--r-- 1 root root 25M Jan 20 09:20 /var/lib/openvas/scap-data/
nvdcve-2.0-2007.xml
-rw-r--r-- 1 root root 31M Jan 20 09:17 /var/lib/openvas/scap-data/
nvdcve-2.0-2008.xml
-rw-r--r-- 1 root root 31M Jan 20 09:14 /var/lib/openvas/scap-data/
nvdcve-2.0-2009.xml
-rw-r--r-- 1 root root 45M Jan 18 09:09 /var/lib/openvas/scap-data/
nvdcve-2.0-2010.xml
-rw-r--r-- 1 root root 109M Jan 7 09:14 /var/lib/openvas/scap-data/
nvdcve-2.0-2011.xml
-rw-r--r-- 1 root root 42M Jan 20 09:12 /var/lib/openvas/scap-data/
nvdcve-2.0-2012.xml
-rw-r--r-- 1 root root 44M Jan 25 09:58 /var/lib/openvas/scap-data/
nvdcve-2.0-2013.xml
-rw-r--r-- 1 root root 40M Jan 25 09:07 /var/lib/openvas/scap-data/
nvdcve-2.0-2014.xml
-rw-r--r-- 1 root root 30M Jan 25 09:04 /var/lib/openvas/scap-data/
nvdcve-2.0-2015.xml
-rw-r--r-- 1 root root 30M Jan 25 09:02 /var/lib/openvas/scap-data/
nvdcve-2.0-2016.xml
-rw-r--r-- 1 root root 442K Jan 25 09:00 /var/lib/openvas/scap-data/
nvdcve-2.0-2017.xml
My question is: what go wrong with sync scap-data and how can I correct it?
Regards,
Michal Chrobak
Michal Chrobak
IT Security Systems Engineer
tel. +48 22 122 09 42
tel. +48 503 555 769
SANSEC Poland S.A. NIP: 7010352299, KRS: 0000429238, REGON: 146270315,
SpÃ³Åka zarejestrowana przez SÄd Rejonowy dla M. St. Warszawy w Warszawie,
XII WydziaÅ Gospodarczy Krajowego Rejestru SÄdowego, KapitaÅ zakÅadowy: 1
000 000 PLN.
Niniejsza wiadomoÅÄ zawiera informacje zastrzeÅŒone i stanowiÄce tajemnicÄ
przedsiÄbiorstwa SANSEC Poland S.A.
_______________________________________________
Openvas-discuss mailing list
https://lists.wald.intevation.org/cgi-bin/mailman/listinfo/openvas-discuss

Michal Chrobak

2017-01-26 14:42:04 UTC

Permalink

As simply as that, ok I try it – for now I have 1GB.
But how can I forced to resync scap-data? When I run openvas-scapdata-sync for second time, it go without errors when parsing /var/lib/openvas/scap-data/nvdcve-2.0-20*xml.

Michal

Michal Chrobak
IT Security Systems Engineer
tel. +48 22 122 09 42
tel. +48 503 555 769From: ***@gmail.com [mailto:***@gmail.com] On Behalf Of Eero Volotinen
Sent: Thursday, January 26, 2017 3:37 PM
To: Michal Chrobak <***@sansec.com>
Cc: openvas-***@wald.intevation.org
Subject: Re: [Openvas-discuss] Fresh install and problem with openvas-scapdata-sync

you need to add more ram memory to machine.

Eero

26.1.2017 4.31 ip. "Michal Chrobak" <mailto:***@sansec.com> kirjoitti:
Hi,

Today I'm trying to install openvas. I downloaded Ubuntu Core 16.04 LTS and install it on KVM virtual machine. Then I add new repository (ppa:mrazavi/openvas from https://launchpad.net/~mrazavi/+archive/ubuntu/openvas) and install openvas 8. Then I run openvas-nvt-sync and openvas-scapdata-sync. And here is my problem, after executing openvas-scap-sync, I've got following error:

***@openvas:~$ sudo openvas-scapdata-sync
[i] This script synchronizes a SCAP data directory with the OpenVAS one.
[i] This script is for the SQLite3 backend.
[i] SCAP dir: /var/lib/openvas/scap-data
[i] Will use rsync
[i] Using rsync: /usr/bin/rsync
[i] Configured SCAP data rsync feed: rsync://feed.openvas.org:/scap-data
OpenVAS community feed server - http://www.openvas.org/
(...)
*rsync cut*
(...)

sent 1,236 bytes received 748,765,593 bytes 815,206.13 bytes/sec
total size is 748,578,705 speedup is 1.00
[i] Initializing scap database
[i] Updating CPEs
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2002.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2003.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2004.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2005.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2006.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2007.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2008.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2009.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2010.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2011.xml
Killed
-:515359: parser error : expected '>'
-:515359: parser error : Premature end of data in tag vulnerable-software-list line 513948
-:515359: parser error : Premature end of data in tag entry line 512501
-:515359: parser error : Premature end of data in tag nvd line 2
unable to parse -
[e] Update of CVEs failed at file '/var/lib/openvas/scap-data/nvdcve-2.0-2011.xml': xsltproc exited with code 137

When I rerun command, it looks that everything is ok, but I'm not convinced that it is true:
***@openvas:~$ sudo openvas-nvt-sync
[i] This script synchronizes an NVT collection with the 'OpenVAS NVT Feed'.
[i] The 'OpenVAS NVT Feed' is provided by 'The OpenVAS Project'.
[i] Online information about this feed: 'http://www.openvas.org/openvas-nvt-feed.html'.
[i] NVT dir: /var/lib/openvas/plugins
OpenVAS community feed server - http://www.openvas.org/
This service is hosted by Greenbone Networks - http://www.greenbone.net/

All transactions are logged.

If you have any questions, please use the OpenVAS mailing lists
or the OpenVAS IRC chat. See http://www.openvas.org/ for details.

By using this service you agree to our terms and conditions.

Only one sync per time, otherwise the source ip will be blocked.

[i] Feed is already current, no synchronization necessary.
***@openvas:~$

My scap-data looks like that:
***@openvas:~$ ls /var/lib/openvas/scap-data/nvdcve-2.0-20*xml -lh
-rw-r--r-- 1 root root 19M Jan 20 09:28 /var/lib/openvas/scap-data/nvdcve-2.0-2002.xml
-rw-r--r-- 1 root root 5.5M Jan 3 09:25 /var/lib/openvas/scap-data/nvdcve-2.0-2003.xml
-rw-r--r-- 1 root root 12M Jan 20 09:26 /var/lib/openvas/scap-data/nvdcve-2.0-2004.xml
-rw-r--r-- 1 root root 18M Jan 20 09:25 /var/lib/openvas/scap-data/nvdcve-2.0-2005.xml
-rw-r--r-- 1 root root 27M Jan 20 09:22 /var/lib/openvas/scap-data/nvdcve-2.0-2006.xml
-rw-r--r-- 1 root root 25M Jan 20 09:20 /var/lib/openvas/scap-data/nvdcve-2.0-2007.xml
-rw-r--r-- 1 root root 31M Jan 20 09:17 /var/lib/openvas/scap-data/nvdcve-2.0-2008.xml
-rw-r--r-- 1 root root 31M Jan 20 09:14 /var/lib/openvas/scap-data/nvdcve-2.0-2009.xml
-rw-r--r-- 1 root root 45M Jan 18 09:09 /var/lib/openvas/scap-data/nvdcve-2.0-2010.xml
-rw-r--r-- 1 root root 109M Jan 7 09:14 /var/lib/openvas/scap-data/nvdcve-2.0-2011.xml
-rw-r--r-- 1 root root 42M Jan 20 09:12 /var/lib/openvas/scap-data/nvdcve-2.0-2012.xml
-rw-r--r-- 1 root root 44M Jan 25 09:58 /var/lib/openvas/scap-data/nvdcve-2.0-2013.xml
-rw-r--r-- 1 root root 40M Jan 25 09:07 /var/lib/openvas/scap-data/nvdcve-2.0-2014.xml
-rw-r--r-- 1 root root 30M Jan 25 09:04 /var/lib/openvas/scap-data/nvdcve-2.0-2015.xml
-rw-r--r-- 1 root root 30M Jan 25 09:02 /var/lib/openvas/scap-data/nvdcve-2.0-2016.xml
-rw-r--r-- 1 root root 442K Jan 25 09:00 /var/lib/openvas/scap-data/nvdcve-2.0-2017.xml

My question is: what go wrong with sync scap-data and how can I correct it?

Regards,
Michal Chrobak

Michal Chrobak
IT Security Systems Engineer
tel. +48 22 122 09 42
tel. +48 503 555 769

SANSEC Poland S.A. NIP: 7010352299, KRS: 0000429238, REGON: 146270315, Spółka zarejestrowana przez Sąd Rejonowy dla M. St. Warszawy w Warszawie, XII Wydział Gospodarczy Krajowego Rejestru Sądowego, Kapitał zakładowy: 1 000 000 PLN.
Niniejsza wiadomość zawiera informacje zastrzeżone i stanowiące tajemnicę przedsiębiorstwa SANSEC Poland S.A.
_______________________________________________
Openvas-discuss mailing list
mailto:Openvas-***@wald.intevation.org
https://lists.wald.intevation.org/cgi-bin/mailman/listinfo/openvas-discuss

Michal Chrobak

2017-01-27 09:40:50 UTC

Permalink

I think its ok for now. What I did:.
1. Increase RAM
2. Run this commands:
***@openvas:~$ mkdir /tmp/scap
***@openvas:~$ sudo mv /var/lib/openvas/scap-data/* /tmp/scap/
***@openvas:~$ sudo openvas-scapdata-sync
[i] This script synchronizes a SCAP data directory with the OpenVAS one.
[i] This script is for the SQLite3 backend.
[i] SCAP dir: /var/lib/openvas/scap-data
[i] Will use rsync
[i] Using rsync: /usr/bin/rsync
[i] Configured SCAP data rsync feed: rsync://feed.openvas.org:/scap-data
OpenVAS community feed server - http://www.openvas.org/
(...rsync...)
[i] Initializing scap database
[i] Updating CPEs
Error: near line 1519808: disk I/O error <<<###########
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2002.xml
(...nvdcve-2.0-2003-20016...)
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2017.xml
[i] Updating OVAL data
[i] Updating /var/lib/openvas/scap-data/oval/5.10/org.mitre.oval/c/oval.xml
(...mitre...)
[i] Updating /var/lib/openvas/scap-data/oval/5.10/org.mitre.oval/v/family/windows.xml
[i] No user data directory '/var/lib/openvas/scap-data/private' found.
[i] Updating CVSS scores and CVE counts for CPEs
[i] Updating CVSS scores for OVAL definitions
[i] Updating placeholder CPEs

And all went ok, but one error show up. My disk:
***@openvas:~$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 981M 0 981M 0% /dev
tmpfs 201M 3.3M 197M 2% /run

/dev/mapper/openvas--vg-root 19G 4.9G 13G 29% /

tmpfs 1001M 0 1001M 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 1001M 0 1001M 0% /sys/fs/cgroup
/dev/vda1 472M 105M 344M 24% /boot
tmpfs 201M 0 201M 0% /run/user/1000

Could be this error important or it happen because I delete everything from /var/lib/openvas/scap-data/* and I can omit it?

Michal

Michal Chrobak
IT Security Systems Engineer
tel. +48 22 122 09 42
tel. +48 503 555 769-----Original Message-----
From: Openvas-discuss [mailto:openvas-discuss-***@wald.intevation.org] On Behalf Of Michal Chrobak
Sent: Thursday, January 26, 2017 3:42 PM
To: Eero Volotinen <***@iki.fi>
Cc: openvas-***@wald.intevation.org
Subject: Re: [Openvas-discuss] Fresh install and problem with openvas-scapdata-sync

As simply as that, ok I try it – for now I have 1GB.
But how can I forced to resync scap-data? When I run openvas-scapdata-sync for second time, it go without errors when parsing /var/lib/openvas/scap-data/nvdcve-2.0-20*xml.

Michal

Michal Chrobak
IT Security Systems Engineer
tel. +48 22 122 09 42
tel. +48 503 555 769From: ***@gmail.com [mailto:***@gmail.com] On Behalf Of Eero Volotinen
Sent: Thursday, January 26, 2017 3:37 PM
To: Michal Chrobak <***@sansec.com>
Cc: openvas-***@wald.intevation.org
Subject: Re: [Openvas-discuss] Fresh install and problem with openvas-scapdata-sync

you need to add more ram memory to machine.

Eero

26.1.2017 4.31 ip. "Michal Chrobak" <mailto:***@sansec.com> kirjoitti:
Hi,

Today I'm trying to install openvas. I downloaded Ubuntu Core 16.04 LTS and install it on KVM virtual machine. Then I add new repository (ppa:mrazavi/openvas from https://launchpad.net/~mrazavi/+archive/ubuntu/openvas) and install openvas 8. Then I run openvas-nvt-sync and openvas-scapdata-sync. And here is my problem, after executing openvas-scap-sync, I've got following error:

***@openvas:~$ sudo openvas-scapdata-sync [i] This script synchronizes a SCAP data directory with the OpenVAS one.
[i] This script is for the SQLite3 backend.
[i] SCAP dir: /var/lib/openvas/scap-data [i] Will use rsync [i] Using rsync: /usr/bin/rsync [i] Configured SCAP data rsync feed: rsync://feed.openvas.org:/scap-data
OpenVAS community feed server - http://www.openvas.org/
(...)
*rsync cut*
(...)

sent 1,236 bytes received 748,765,593 bytes 815,206.13 bytes/sec total size is 748,578,705 speedup is 1.00 [i] Initializing scap database [i] Updating CPEs [i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2002.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2003.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2004.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2005.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2006.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2007.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2008.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2009.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2010.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2011.xml
Killed
-:515359: parser error : expected '>'
-:515359: parser error : Premature end of data in tag vulnerable-software-list line 513948
-:515359: parser error : Premature end of data in tag entry line 512501
-:515359: parser error : Premature end of data in tag nvd line 2 unable to parse - [e] Update of CVEs failed at file '/var/lib/openvas/scap-data/nvdcve-2.0-2011.xml': xsltproc exited with code 137

When I rerun command, it looks that everything is ok, but I'm not convinced that it is true:
***@openvas:~$ sudo openvas-nvt-sync
[i] This script synchronizes an NVT collection with the 'OpenVAS NVT Feed'.
[i] The 'OpenVAS NVT Feed' is provided by 'The OpenVAS Project'.
[i] Online information about this feed: 'http://www.openvas.org/openvas-nvt-feed.html'.
[i] NVT dir: /var/lib/openvas/plugins
OpenVAS community feed server - http://www.openvas.org/ This service is hosted by Greenbone Networks - http://www.greenbone.net/

All transactions are logged.

If you have any questions, please use the OpenVAS mailing lists or the OpenVAS IRC chat. See http://www.openvas.org/ for details.

By using this service you agree to our terms and conditions.

Only one sync per time, otherwise the source ip will be blocked.

[i] Feed is already current, no synchronization necessary.
***@openvas:~$

My scap-data looks like that:
***@openvas:~$ ls /var/lib/openvas/scap-data/nvdcve-2.0-20*xml -lh
-rw-r--r-- 1 root root 19M Jan 20 09:28 /var/lib/openvas/scap-data/nvdcve-2.0-2002.xml
-rw-r--r-- 1 root root 5.5M Jan 3 09:25 /var/lib/openvas/scap-data/nvdcve-2.0-2003.xml
-rw-r--r-- 1 root root 12M Jan 20 09:26 /var/lib/openvas/scap-data/nvdcve-2.0-2004.xml
-rw-r--r-- 1 root root 18M Jan 20 09:25 /var/lib/openvas/scap-data/nvdcve-2.0-2005.xml
-rw-r--r-- 1 root root 27M Jan 20 09:22 /var/lib/openvas/scap-data/nvdcve-2.0-2006.xml
-rw-r--r-- 1 root root 25M Jan 20 09:20 /var/lib/openvas/scap-data/nvdcve-2.0-2007.xml
-rw-r--r-- 1 root root 31M Jan 20 09:17 /var/lib/openvas/scap-data/nvdcve-2.0-2008.xml
-rw-r--r-- 1 root root 31M Jan 20 09:14 /var/lib/openvas/scap-data/nvdcve-2.0-2009.xml
-rw-r--r-- 1 root root 45M Jan 18 09:09 /var/lib/openvas/scap-data/nvdcve-2.0-2010.xml
-rw-r--r-- 1 root root 109M Jan 7 09:14 /var/lib/openvas/scap-data/nvdcve-2.0-2011.xml
-rw-r--r-- 1 root root 42M Jan 20 09:12 /var/lib/openvas/scap-data/nvdcve-2.0-2012.xml
-rw-r--r-- 1 root root 44M Jan 25 09:58 /var/lib/openvas/scap-data/nvdcve-2.0-2013.xml
-rw-r--r-- 1 root root 40M Jan 25 09:07 /var/lib/openvas/scap-data/nvdcve-2.0-2014.xml
-rw-r--r-- 1 root root 30M Jan 25 09:04 /var/lib/openvas/scap-data/nvdcve-2.0-2015.xml
-rw-r--r-- 1 root root 30M Jan 25 09:02 /var/lib/openvas/scap-data/nvdcve-2.0-2016.xml
-rw-r--r-- 1 root root 442K Jan 25 09:00 /var/lib/openvas/scap-data/nvdcve-2.0-2017.xml

My question is: what go wrong with sync scap-data and how can I correct it?

Regards,
Michal Chrobak

Michal Chrobak
IT Security Systems Engineer
tel. +48 22 122 09 42
tel. +48 503 555 769

SANSEC Poland S.A. NIP: 7010352299, KRS: 0000429238, REGON: 146270315, Spółka zarejestrowana przez Sąd Rejonowy dla M. St. Warszawy w Warszawie, XII Wydział Gospodarczy Krajowego Rejestru Sądowego, Kapitał zakładowy: 1 000 000 PLN.
Niniejsza wiadomość zawiera informacje zastrzeżone i stanowiące tajemnicę przedsiębiorstwa SANSEC Poland S.A.
_______________________________________________
Openvas-discuss mailing list
mailto:Openvas-***@wald.intevation.org
https://lists.wald.intevation.org/cgi-bin/mailman/listinfo/openvas-discuss
_______________________________________________
Openvas-discuss mailing list
Openvas-***@wald.intevation.org
https://lists.wald.intevation.org/cgi-bin/mailman/listinfo/openvas-discuss

Eero Volotinen

2017-01-27 09:45:31 UTC

Permalink

disk io error sounds scary. is your harddisk broken? replace with working
on and try again?

Eero

Post by Michal Chrobak
I think its ok for now. What I did:.
1. Increase RAM
[i] This script synchronizes a SCAP data directory with the OpenVAS one.
[i] This script is for the SQLite3 backend.
[i] SCAP dir: /var/lib/openvas/scap-data
[i] Will use rsync
[i] Using rsync: /usr/bin/rsync
[i] Configured SCAP data rsync feed: rsync://feed.openvas.org:/scap-data
OpenVAS community feed server - http://www.openvas.org/
(...rsync...)
[i] Initializing scap database
[i] Updating CPEs
Error: near line 1519808: disk I/O error <<<###########
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2002.xml
(...nvdcve-2.0-2003-20016...)
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2017.xml
[i] Updating OVAL data
[i] Updating /var/lib/openvas/scap-data/oval/5.10/org.mitre.oval/c/
oval.xml
(...mitre...)
[i] Updating /var/lib/openvas/scap-data/oval/5.10/org.mitre.oval/v/
family/windows.xml
[i] No user data directory '/var/lib/openvas/scap-data/private' found.
[i] Updating CVSS scores and CVE counts for CPEs
[i] Updating CVSS scores for OVAL definitions
[i] Updating placeholder CPEs
Filesystem Size Used Avail Use% Mounted on
udev 981M 0 981M 0% /dev
tmpfs 201M 3.3M 197M 2% /run
/dev/mapper/openvas--vg-root 19G 4.9G 13G 29% /
tmpfs 1001M 0 1001M 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 1001M 0 1001M 0% /sys/fs/cgroup
/dev/vda1 472M 105M 344M 24% /boot
tmpfs 201M 0 201M 0% /run/user/1000
Could be this error important or it happen because I delete everything
from /var/lib/openvas/scap-data/* and I can omit it?
Michal
Michal Chrobak
IT Security Systems Engineer
tel. +48 22 122 09 42
tel. +48 503 555 769-----Original Message-----
On Behalf Of Michal Chrobak
Sent: Thursday, January 26, 2017 3:42 PM
Subject: Re: [Openvas-discuss] Fresh install and problem with
openvas-scapdata-sync
As simply as that, ok I try it â for now I have 1GB.
But how can I forced to resync scap-data? When I run openvas-scapdata-sync
for second time, it go without errors when parsing
/var/lib/openvas/scap-data/nvdcve-2.0-20*xml.
Michal
Michal Chrobak
IT Security Systems Engineer
tel. +48 22 122 09 42
Sent: Thursday, January 26, 2017 3:37 PM
Subject: Re: [Openvas-discuss] Fresh install and problem with
openvas-scapdata-sync
you need to add more ram memory to machine.
Eero
Hi,
Today I'm trying to install openvas. I downloaded Ubuntu Core 16.04 LTS
and install it on KVM virtual machine. Then I add new repository
(ppa:mrazavi/openvas from https://launchpad.net/~mrazavi/+archive/ubuntu/
openvas) and install openvas 8. Then I run openvas-nvt-sync and
openvas-scapdata-sync. And here is my problem, after executing
SCAP data directory with the OpenVAS one.
[i] This script is for the SQLite3 backend.
[i] SCAP dir: /var/lib/openvas/scap-data [i] Will use rsync [i] Using
rsync://feed.openvas.org:/scap-data
OpenVAS community feed server - http://www.openvas.org/
(...)
*rsync cut*
(...)
sent 1,236 bytes received 748,765,593 bytes 815,206.13 bytes/sec total
size is 748,578,705 speedup is 1.00 [i] Initializing scap database [i]
Updating CPEs [i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2002.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2003.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2004.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2005.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2006.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2007.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2008.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2009.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2010.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2011.xml
Killed
-:515359: parser error : expected '>'
-:515359: parser error : Premature end of data in tag
vulnerable-software-list line 513948
-:515359: parser error : Premature end of data in tag entry line 512501
-:515359: parser error : Premature end of data in tag nvd line 2 unable to
xsltproc exited with code 137
When I rerun command, it looks that everything is ok, but I'm not
[i] This script synchronizes an NVT collection with the 'OpenVAS NVT Feed'.
[i] The 'OpenVAS NVT Feed' is provided by 'The OpenVAS Project'.
[i] Online information about this feed: 'http://www.openvas.org/
openvas-nvt-feed.html'.
[i] NVT dir: /var/lib/openvas/plugins
OpenVAS community feed server - http://www.openvas.org/ This service is
hosted by Greenbone Networks - http://www.greenbone.net/
All transactions are logged.
If you have any questions, please use the OpenVAS mailing lists or the
OpenVAS IRC chat. See http://www.openvas.org/ for details.
By using this service you agree to our terms and conditions.
Only one sync per time, otherwise the source ip will be blocked.
[i] Feed is already current, no synchronization necessary.
-rw-r--r-- 1 root root 19M Jan 20 09:28 /var/lib/openvas/scap-data/
nvdcve-2.0-2002.xml
-rw-r--r-- 1 root root 5.5M Jan 3 09:25 /var/lib/openvas/scap-data/
nvdcve-2.0-2003.xml
-rw-r--r-- 1 root root 12M Jan 20 09:26 /var/lib/openvas/scap-data/
nvdcve-2.0-2004.xml
-rw-r--r-- 1 root root 18M Jan 20 09:25 /var/lib/openvas/scap-data/
nvdcve-2.0-2005.xml
-rw-r--r-- 1 root root 27M Jan 20 09:22 /var/lib/openvas/scap-data/
nvdcve-2.0-2006.xml
-rw-r--r-- 1 root root 25M Jan 20 09:20 /var/lib/openvas/scap-data/
nvdcve-2.0-2007.xml
-rw-r--r-- 1 root root 31M Jan 20 09:17 /var/lib/openvas/scap-data/
nvdcve-2.0-2008.xml
-rw-r--r-- 1 root root 31M Jan 20 09:14 /var/lib/openvas/scap-data/
nvdcve-2.0-2009.xml
-rw-r--r-- 1 root root 45M Jan 18 09:09 /var/lib/openvas/scap-data/
nvdcve-2.0-2010.xml
-rw-r--r-- 1 root root 109M Jan 7 09:14 /var/lib/openvas/scap-data/
nvdcve-2.0-2011.xml
-rw-r--r-- 1 root root 42M Jan 20 09:12 /var/lib/openvas/scap-data/
nvdcve-2.0-2012.xml
-rw-r--r-- 1 root root 44M Jan 25 09:58 /var/lib/openvas/scap-data/
nvdcve-2.0-2013.xml
-rw-r--r-- 1 root root 40M Jan 25 09:07 /var/lib/openvas/scap-data/
nvdcve-2.0-2014.xml
-rw-r--r-- 1 root root 30M Jan 25 09:04 /var/lib/openvas/scap-data/
nvdcve-2.0-2015.xml
-rw-r--r-- 1 root root 30M Jan 25 09:02 /var/lib/openvas/scap-data/
nvdcve-2.0-2016.xml
-rw-r--r-- 1 root root 442K Jan 25 09:00 /var/lib/openvas/scap-data/
nvdcve-2.0-2017.xml
My question is: what go wrong with sync scap-data and how can I correct it?
Regards,
Michal Chrobak
Michal Chrobak
IT Security Systems Engineer
tel. +48 22 122 09 42
tel. +48 503 555 769
SANSEC Poland S.A. NIP: 7010352299, KRS: 0000429238, REGON: 146270315,
SpÃ³Åka zarejestrowana przez SÄd Rejonowy dla M. St. Warszawy w Warszawie,
XII WydziaÅ Gospodarczy Krajowego Rejestru SÄdowego, KapitaÅ zakÅadowy: 1
000 000 PLN.
Niniejsza wiadomoÅÄ zawiera informacje zastrzeÅŒone i stanowiÄce tajemnicÄ
przedsiÄbiorstwa SANSEC Poland S.A.
_______________________________________________
Openvas-discuss mailing list
https://lists.wald.intevation.org/cgi-bin/mailman/listinfo/openvas-discuss
_______________________________________________
Openvas-discuss mailing list
https://lists.wald.intevation.org/cgi-bin/mailman/listinfo/openvas-discuss

Michal Chrobak

2017-01-27 10:20:16 UTC

Permalink

I have my openvas installation on ubuntu which is virtual machine (hosted od arch and kvm) with vmdk disk. I check this disk with badblocks:
***@openvas:~$ sudo poweroff
Connection to 192.168.121.253 closed by remote host.
Connection to 192.168.121.253 closed.
[***@michalc_lin] ~ $ sudo badblocks -nvs kvm/openvas.vmdk
Checking for bad blocks in non-destructive read-write mode
From block 0 to 7808703
Checking for bad blocks (non-destructive read-write test)
Testing with random pattern: done
Pass completed, 0 bad blocks found. (0/0/0 errors)

It looks like vmdk and disk of host OS is ok (I have SSD disk, buy year ago).

Michal

Michal Chrobak
IT Security Systems Engineer
tel. +48 22 122 09 42
tel. +48 503 555 769From: ***@gmail.com [mailto:***@gmail.com] On Behalf Of Eero Volotinen
Sent: Friday, January 27, 2017 10:46 AM
To: Michal Chrobak <***@sansec.com>
Cc: openvas-***@wald.intevation.org
Subject: RE: [Openvas-discuss] Fresh install and problem with openvas-scapdata-sync

disk io error sounds scary. is your harddisk broken? replace with working on and try again?

Eero

27.1.2017 11.40 ap. "Michal Chrobak" <mailto:***@sansec.com> kirjoitti:
I think its ok for now. What I did:.
1. Increase RAM
2. Run this commands:
***@openvas:~$ mkdir /tmp/scap
***@openvas:~$ sudo mv /var/lib/openvas/scap-data/* /tmp/scap/
***@openvas:~$ sudo openvas-scapdata-sync
[i] This script synchronizes a SCAP data directory with the OpenVAS one.
[i] This script is for the SQLite3 backend.
[i] SCAP dir: /var/lib/openvas/scap-data
[i] Will use rsync
[i] Using rsync: /usr/bin/rsync
[i] Configured SCAP data rsync feed: rsync://feed.openvas.org:/scap-data
OpenVAS community feed server - http://www.openvas.org/
(...rsync...)
[i] Initializing scap database
[i] Updating CPEs
Error: near line 1519808: disk I/O error <<<###########
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2002.xml
(...nvdcve-2.tel:0-2003-20016...)
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2017.xml
[i] Updating OVAL data
[i] Updating /var/lib/openvas/scap-data/oval/5.10/org.mitre.oval/c/oval.xml
(...mitre...)
[i] Updating /var/lib/openvas/scap-data/oval/5.10/org.mitre.oval/v/family/windows.xml
[i] No user data directory '/var/lib/openvas/scap-data/private' found.
[i] Updating CVSS scores and CVE counts for CPEs
[i] Updating CVSS scores for OVAL definitions
[i] Updating placeholder CPEs

And all went ok, but one error show up. My disk:
***@openvas:~$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 981M 0 981M 0% /dev
tmpfs 201M 3.3M 197M 2% /run

/dev/mapper/openvas--vg-root 19G 4.9G 13G 29% /

tmpfs 1001M 0 1001M 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 1001M 0 1001M 0% /sys/fs/cgroup
/dev/vda1 472M 105M 344M 24% /boot
tmpfs 201M 0 201M 0% /run/user/1000

Could be this error important or it happen because I delete everything from /var/lib/openvas/scap-data/* and I can omit it?

Michal

Michal Chrobak
IT Security Systems Engineer
tel. +48 22 122 09 42
tel. +48 503 555 769-----Original Message-----
From: Openvas-discuss [mailto:mailto:openvas-discuss-***@wald.intevation.org] On Behalf Of Michal Chrobak
Sent: Thursday, January 26, 2017 3:42 PM
To: Eero Volotinen <mailto:***@iki.fi>
Cc: mailto:openvas-***@wald.intevation.org
Subject: Re: [Openvas-discuss] Fresh install and problem with openvas-scapdata-sync

As simply as that, ok I try it – for now I have 1GB.
But how can I forced to resync scap-data? When I run openvas-scapdata-sync for second time, it go without errors when parsing /var/lib/openvas/scap-data/nvdcve-2.0-20*xml.

Michal

Michal Chrobak
IT Security Systems Engineer
tel. tel:%2B48%2022%20122%2009%2042
tel. +48 503 555 769From: mailto:***@gmail.com [mailto:mailto:***@gmail.com] On Behalf Of Eero Volotinen
Sent: Thursday, January 26, 2017 3:37 PM
To: Michal Chrobak <mailto:***@sansec.com>
Cc: mailto:openvas-***@wald.intevation.org
Subject: Re: [Openvas-discuss] Fresh install and problem with openvas-scapdata-sync

you need to add more ram memory to machine.

Eero

26.1.2017 4.31 ip. "Michal Chrobak" <mailto:mailto:***@sansec.com> kirjoitti:
Hi,

Today I'm trying to install openvas. I downloaded Ubuntu Core 16.04 LTS and install it on KVM virtual machine. Then I add new repository (ppa:mrazavi/openvas from https://launchpad.net/~mrazavi/+archive/ubuntu/openvas) and install openvas 8. Then I run openvas-nvt-sync and openvas-scapdata-sync. And here is my problem, after executing openvas-scap-sync, I've got following error:

***@openvas:~$ sudo openvas-scapdata-sync [i] This script synchronizes a SCAP data directory with the OpenVAS one.
[i] This script is for the SQLite3 backend.
[i] SCAP dir: /var/lib/openvas/scap-data [i] Will use rsync [i] Using rsync: /usr/bin/rsync [i] Configured SCAP data rsync feed: rsync://feed.openvas.org:/scap-data
OpenVAS community feed server - http://www.openvas.org/
(...)
*rsync cut*
(...)

sent 1,236 bytes received 748,765,593 bytes 815,206.13 bytes/sec total size is 748,578,705 speedup is 1.00 [i] Initializing scap database [i] Updating CPEs [i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2002.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2003.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2004.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2005.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2006.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2007.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2008.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2009.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2010.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2011.xml
Killed
-:515359: parser error : expected '>'
-:515359: parser error : Premature end of data in tag vulnerable-software-list line 513948
-:515359: parser error : Premature end of data in tag entry line 512501
-:515359: parser error : Premature end of data in tag nvd line 2 unable to parse - [e] Update of CVEs failed at file '/var/lib/openvas/scap-data/nvdcve-2.0-2011.xml': xsltproc exited with code 137

When I rerun command, it looks that everything is ok, but I'm not convinced that it is true:
***@openvas:~$ sudo openvas-nvt-sync
[i] This script synchronizes an NVT collection with the 'OpenVAS NVT Feed'.
[i] The 'OpenVAS NVT Feed' is provided by 'The OpenVAS Project'.
[i] Online information about this feed: 'http://www.openvas.org/openvas-nvt-feed.html'.
[i] NVT dir: /var/lib/openvas/plugins
OpenVAS community feed server - http://www.openvas.org/ This service is hosted by Greenbone Networks - http://www.greenbone.net/

All transactions are logged.

If you have any questions, please use the OpenVAS mailing lists or the OpenVAS IRC chat. See http://www.openvas.org/ for details.

By using this service you agree to our terms and conditions.

Only one sync per time, otherwise the source ip will be blocked.

[i] Feed is already current, no synchronization necessary.
***@openvas:~$

My scap-data looks like that:
***@openvas:~$ ls /var/lib/openvas/scap-data/nvdcve-2.0-20*xml -lh
-rw-r--r-- 1 root root 19M Jan 20 09:28 /var/lib/openvas/scap-data/nvdcve-2.0-2002.xml
-rw-r--r-- 1 root root 5.5M Jan 3 09:25 /var/lib/openvas/scap-data/nvdcve-2.0-2003.xml
-rw-r--r-- 1 root root 12M Jan 20 09:26 /var/lib/openvas/scap-data/nvdcve-2.0-2004.xml
-rw-r--r-- 1 root root 18M Jan 20 09:25 /var/lib/openvas/scap-data/nvdcve-2.0-2005.xml
-rw-r--r-- 1 root root 27M Jan 20 09:22 /var/lib/openvas/scap-data/nvdcve-2.0-2006.xml
-rw-r--r-- 1 root root 25M Jan 20 09:20 /var/lib/openvas/scap-data/nvdcve-2.0-2007.xml
-rw-r--r-- 1 root root 31M Jan 20 09:17 /var/lib/openvas/scap-data/nvdcve-2.0-2008.xml
-rw-r--r-- 1 root root 31M Jan 20 09:14 /var/lib/openvas/scap-data/nvdcve-2.0-2009.xml
-rw-r--r-- 1 root root 45M Jan 18 09:09 /var/lib/openvas/scap-data/nvdcve-2.0-2010.xml
-rw-r--r-- 1 root root 109M Jan 7 09:14 /var/lib/openvas/scap-data/nvdcve-2.0-2011.xml
-rw-r--r-- 1 root root 42M Jan 20 09:12 /var/lib/openvas/scap-data/nvdcve-2.0-2012.xml
-rw-r--r-- 1 root root 44M Jan 25 09:58 /var/lib/openvas/scap-data/nvdcve-2.0-2013.xml
-rw-r--r-- 1 root root 40M Jan 25 09:07 /var/lib/openvas/scap-data/nvdcve-2.0-2014.xml
-rw-r--r-- 1 root root 30M Jan 25 09:04 /var/lib/openvas/scap-data/nvdcve-2.0-2015.xml
-rw-r--r-- 1 root root 30M Jan 25 09:02 /var/lib/openvas/scap-data/nvdcve-2.0-2016.xml
-rw-r--r-- 1 root root 442K Jan 25 09:00 /var/lib/openvas/scap-data/nvdcve-2.0-2017.xml

My question is: what go wrong with sync scap-data and how can I correct it?

Regards,
Michal Chrobak

Michal Chrobak
IT Security Systems Engineer
tel. +48 22 122 09 42
tel. +48 503 555 769

SANSEC Poland S.A. NIP: 7010352299, KRS: 0000429238, REGON: 146270315, Spółka zarejestrowana przez Sąd Rejonowy dla M. St. Warszawy w Warszawie, XII Wydział Gospodarczy Krajowego Rejestru Sądowego, Kapitał zakładowy: 1 000 000 PLN.
Niniejsza wiadomość zawiera informacje zastrzeżone i stanowiące tajemnicę przedsiębiorstwa SANSEC Poland S.A.
_______________________________________________
Openvas-discuss mailing list
mailto:mailto:Openvas-***@wald.intevation.org
https://lists.wald.intevation.org/cgi-bin/mailman/listinfo/openvas-discuss
_______________________________________________
Openvas-discuss mailing list
mailto:Openvas-***@wald.intevation.org
https://lists.wald.intevation.org/cgi-bin/mailman/listinfo/openvas-discuss

Eero Volotinen

2017-01-27 10:23:14 UTC

Permalink

well, then try again.

If you still get disk io error then there is something wrong with your
machine or filesystem (corrupted?)

--
Eero

Post by Michal Chrobak
I have my openvas installation on ubuntu which is virtual machine (hosted
Connection to 192.168.121.253 closed by remote host.
Connection to 192.168.121.253 closed.
Checking for bad blocks in non-destructive read-write mode
From block 0 to 7808703
Checking for bad blocks (non-destructive read-write test)
Testing with random pattern: done
Pass completed, 0 bad blocks found. (0/0/0 errors)
It looks like vmdk and disk of host OS is ok (I have SSD disk, buy year ago).
Michal
Michal Chrobak
IT Security Systems Engineer
tel. +48 22 122 09 42
Sent: Friday, January 27, 2017 10:46 AM
Subject: RE: [Openvas-discuss] Fresh install and problem with
openvas-scapdata-sync
disk io error sounds scary. is your harddisk broken? replace with working on and try again?
Eero
I think its ok for now. What I did:.
1. Increase RAM
[i] This script synchronizes a SCAP data directory with the OpenVAS one.
[i] This script is for the SQLite3 backend.
[i] SCAP dir: /var/lib/openvas/scap-data
[i] Will use rsync
[i] Using rsync: /usr/bin/rsync
[i] Configured SCAP data rsync feed: rsync://feed.openvas.org:/scap-data
OpenVAS community feed server - http://www.openvas.org/
(...rsync...)
[i] Initializing scap database
[i] Updating CPEs
Error: near line 1519808: disk I/O error <<<###########
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2002.xml
(...nvdcve-2.tel:0-2003-20016...)
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2017.xml
[i] Updating OVAL data
[i] Updating /var/lib/openvas/scap-data/oval/5.10/org.mitre.oval/c/
oval.xml
(...mitre...)
[i] Updating /var/lib/openvas/scap-data/oval/5.10/org.mitre.oval/v/
family/windows.xml
[i] No user data directory '/var/lib/openvas/scap-data/private' found.
[i] Updating CVSS scores and CVE counts for CPEs
[i] Updating CVSS scores for OVAL definitions
[i] Updating placeholder CPEs
Filesystem Size Used Avail Use% Mounted on
udev 981M 0 981M 0% /dev
tmpfs 201M 3.3M 197M 2% /run
/dev/mapper/openvas--vg-root 19G 4.9G 13G 29% /
tmpfs 1001M 0 1001M 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 1001M 0 1001M 0% /sys/fs/cgroup
/dev/vda1 472M 105M 344M 24% /boot
tmpfs 201M 0 201M 0% /run/user/1000
Could be this error important or it happen because I delete everything
from /var/lib/openvas/scap-data/* and I can omit it?
Michal
Michal Chrobak
IT Security Systems Engineer
tel. +48 22 122 09 42
tel. +48 503 555 769-----Original Message-----
intevation.org] On Behalf Of Michal Chrobak
Sent: Thursday, January 26, 2017 3:42 PM
Subject: Re: [Openvas-discuss] Fresh install and problem with
openvas-scapdata-sync
As simply as that, ok I try it â for now I have 1GB.
But how can I forced to resync scap-data? When I run openvas-scapdata-sync
for second time, it go without errors when parsing
/var/lib/openvas/scap-data/nvdcve-2.0-20*xml.
Michal
Michal Chrobak
IT Security Systems Engineer
tel. tel:%2B48%2022%20122%2009%2042
Sent: Thursday, January 26, 2017 3:37 PM
Subject: Re: [Openvas-discuss] Fresh install and problem with
openvas-scapdata-sync
you need to add more ram memory to machine.
Eero
Hi,
Today I'm trying to install openvas. I downloaded Ubuntu Core 16.04 LTS
and install it on KVM virtual machine. Then I add new repository
(ppa:mrazavi/openvas from https://launchpad.net/~mrazavi/+archive/ubuntu/
openvas) and install openvas 8. Then I run openvas-nvt-sync and
openvas-scapdata-sync. And here is my problem, after executing
SCAP data directory with the OpenVAS one.
[i] This script is for the SQLite3 backend.
[i] SCAP dir: /var/lib/openvas/scap-data [i] Will use rsync [i] Using
rsync://feed.openvas.org:/scap-data
OpenVAS community feed server - http://www.openvas.org/
(...)
*rsync cut*
(...)
sent 1,236 bytes received 748,765,593 bytes 815,206.13 bytes/sec total
size is 748,578,705 speedup is 1.00 [i] Initializing scap database [i]
Updating CPEs [i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2002.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2003.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2004.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2005.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2006.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2007.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2008.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2009.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2010.xml
[i] Updating /var/lib/openvas/scap-data/nvdcve-2.0-2011.xml
Killed
-:515359: parser error : expected '>'
-:515359: parser error : Premature end of data in tag
vulnerable-software-list line 513948
-:515359: parser error : Premature end of data in tag entry line 512501
-:515359: parser error : Premature end of data in tag nvd line 2 unable to
xsltproc exited with code 137
When I rerun command, it looks that everything is ok, but I'm not
[i] This script synchronizes an NVT collection with the 'OpenVAS NVT Feed'.
[i] The 'OpenVAS NVT Feed' is provided by 'The OpenVAS Project'.
[i] Online information about this feed: 'http://www.openvas.org/
openvas-nvt-feed.html'.
[i] NVT dir: /var/lib/openvas/plugins
OpenVAS community feed server - http://www.openvas.org/ This service is
hosted by Greenbone Networks - http://www.greenbone.net/
All transactions are logged.
If you have any questions, please use the OpenVAS mailing lists or the
OpenVAS IRC chat. See http://www.openvas.org/ for details.
By using this service you agree to our terms and conditions.
Only one sync per time, otherwise the source ip will be blocked.
[i] Feed is already current, no synchronization necessary.
-rw-r--r-- 1 root root 19M Jan 20 09:28 /var/lib/openvas/scap-data/
nvdcve-2.0-2002.xml
-rw-r--r-- 1 root root 5.5M Jan 3 09:25 /var/lib/openvas/scap-data/
nvdcve-2.0-2003.xml
-rw-r--r-- 1 root root 12M Jan 20 09:26 /var/lib/openvas/scap-data/
nvdcve-2.0-2004.xml
-rw-r--r-- 1 root root 18M Jan 20 09:25 /var/lib/openvas/scap-data/
nvdcve-2.0-2005.xml
-rw-r--r-- 1 root root 27M Jan 20 09:22 /var/lib/openvas/scap-data/
nvdcve-2.0-2006.xml
-rw-r--r-- 1 root root 25M Jan 20 09:20 /var/lib/openvas/scap-data/
nvdcve-2.0-2007.xml
-rw-r--r-- 1 root root 31M Jan 20 09:17 /var/lib/openvas/scap-data/
nvdcve-2.0-2008.xml
-rw-r--r-- 1 root root 31M Jan 20 09:14 /var/lib/openvas/scap-data/
nvdcve-2.0-2009.xml
-rw-r--r-- 1 root root 45M Jan 18 09:09 /var/lib/openvas/scap-data/
nvdcve-2.0-2010.xml
-rw-r--r-- 1 root root 109M Jan 7 09:14 /var/lib/openvas/scap-data/
nvdcve-2.0-2011.xml
-rw-r--r-- 1 root root 42M Jan 20 09:12 /var/lib/openvas/scap-data/
nvdcve-2.0-2012.xml
-rw-r--r-- 1 root root 44M Jan 25 09:58 /var/lib/openvas/scap-data/
nvdcve-2.0-2013.xml
-rw-r--r-- 1 root root 40M Jan 25 09:07 /var/lib/openvas/scap-data/
nvdcve-2.0-2014.xml
-rw-r--r-- 1 root root 30M Jan 25 09:04 /var/lib/openvas/scap-data/
nvdcve-2.0-2015.xml
-rw-r--r-- 1 root root 30M Jan 25 09:02 /var/lib/openvas/scap-data/
nvdcve-2.0-2016.xml
-rw-r--r-- 1 root root 442K Jan 25 09:00 /var/lib/openvas/scap-data/
nvdcve-2.0-2017.xml
My question is: what go wrong with sync scap-data and how can I correct it?
Regards,
Michal Chrobak
Michal Chrobak
IT Security Systems Engineer
tel. +48 22 122 09 42
tel. +48 503 555 769
SANSEC Poland S.A. NIP: 7010352299, KRS: 0000429238, REGON: 146270315,
SpÃ³Åka zarejestrowana przez SÄd Rejonowy dla M. St. Warszawy w Warszawie,
XII WydziaÅ Gospodarczy Krajowego Rejestru SÄdowego, KapitaÅ zakÅadowy: 1
000 000 PLN.
Niniejsza wiadomoÅÄ zawiera informacje zastrzeÅŒone i stanowiÄce tajemnicÄ
przedsiÄbiorstwa SANSEC Poland S.A.
_______________________________________________
Openvas-discuss mailing list
https://lists.wald.intevation.org/cgi-bin/mailman/listinfo/openvas-discuss
_______________________________________________
Openvas-discuss mailing list
https://lists.wald.intevation.org/cgi-bin/mailman/listinfo/openvas-discuss

Reindl Harald

2017-01-27 14:24:49 UTC

Permalink

Post by Michal Chrobak
Connection to 192.168.121.253 closed by remote host.
Connection to 192.168.121.253 closed.
Checking for bad blocks in non-destructive read-write mode
From block 0 to 7808703
Checking for bad blocks (non-destructive read-write test)
Testing with random pattern: done
Pass completed, 0 bad blocks found. (0/0/0 errors)
It looks like vmdk and disk of host OS is ok (I have SSD disk, buy year ago)

frankly "badblocks" on a virtual disk image should dot what exactly?

what about read the output of "dmesg" and whereever Ubunto writes the
global syslog at times where probles appear?

look in your systemlogs on the host and the virtual machine, in the best
case the was only concurrency from other guests and you would see
something like "[ 1985.288632] hrtimer: interrupt took 2895550 ns" or
disk timouts in the guest

disk timeouts are happening here regulary when the host does it's weekly
raid-check and the backup vm it's monthly rsync --checksums over 1 TB
btrfs compressed data on a LUKS encryption layer at the same time

but that's not the production host

Michal Chrobak

2017-01-27 15:08:26 UTC

Permalink

Michal Chrobak
IT Security Systems Engineer
tel. +48 22 122 09 42
tel. +48 503 555 769

SANSEC Poland S.A. NIP: 7010352299, KRS: 0000429238, REGON: 146270315, Spółka zarejestrowana przez Sąd Rejonowy dla M. St. Warszawy w Warszawie, XII Wydział Gospodarczy Krajowego Rejestru Sądowego, Kapitał zakładowy: 1 000 000 PLN.
Niniejsza wiadomość zawiera informacje zastrzeżone i stanowiące tajemnicę przedsiębiorstwa SANSEC Poland S.A.

Post by Reindl Harald

Post by Michal Chrobak
Connection to 192.168.121.253 closed by remote host.
Connection to 192.168.121.253 closed.
Checking for bad blocks in non-destructive read-write mode From block
0 to 7808703 Checking for bad blocks (non-destructive read-write test)
Testing with random pattern: done Pass completed, 0 bad blocks found.
(0/0/0 errors)
It looks like vmdk and disk of host OS is ok (I have SSD disk, buy year ago)

frankly "badblocks" on a virtual disk image should dot what exactly?
what about read the output of "dmesg" and whereever Ubunto writes the global syslog at times where probles appear?
look in your systemlogs on the host and the virtual machine, in the best case the was only concurrency from other guests and you would see something like "[ 1985.288632] hrtimer: interrupt took 2895550 ns" or disk timouts in the guest
disk timeouts are happening here regulary when the host does it's weekly raid-check and the backup vm it's monthly rsync --checksums over 1 TB btrfs compressed data on a LUKS encryption layer at the same time
but that's not the production host

I used badblocks to check part of my physical disk where is openvas.vmdk.

I run dmesg -w on guest os and host os, when executing openvas-scap-sync again. No logs reported.

VM with openvas is on my notebook, where I also have second running VM (Win10). Some time ago I was checking my disk i/o with munin and this wasn't my bottleneck.

What I want do next, is change disk format from vmdh to qcow2/raw and/or change kvm disk driver from VirtIO to IDE/SATA.