upgrade complete
Saving upgrade log to /var/log/noc/upgrade.log
[16:02:49][root@s-5-bak71][noc]#
[16:03:47][root@s-5-bak71][noc]#
[16:03:47][root@s-5-bak71][noc]#
[16:03:47][root@s-5-bak71][noc]# ./noc fix
Checking collections UUID
... done
Checking inventory Root
Checking inventory Lost&Found
Checking pm.MetricSettings
... done
Checking fm.Outages
Restarting wipe process: wiping-2
[16:03:53][root@s-5-bak71][noc]# ./noc fix
Checking collections UUID
... done
Checking inventory Root
Checking inventory Lost&Found
Checking pm.MetricSettings
... done
Checking fm.Outages
Restarting wipe process: wiping-2
[16:03:59][root@s-5-bak71][noc]# ./noc wipe managed-object wiping-2
Wiping 'wiping-2':
ERROR:noc.lib.debug:UNHANDLED EXCEPTION (2015-05-18 16:04:13.505909)
BRANCH: develop TIP: 8f4239693e88
PROCESS: manage.py
ERROR FINGERPRINT: ca6e7207-7bc6-5f5a-b35d-6565755ffad7
WORKING DIRECTORY: /usr/local/noc
EXCEPTION: <class 'noc.sa.models.managedobject.DoesNotExist'> ManagedObject matching query does not exist.
START OF TRACEBACK
------------------------------------------------------------------------
File: lib/python2.7/site-packages/django/db/models/query.py (Line: 366)
Function: get
  359             if self.query.can_filter():
  360                 clone = clone.order_by()
  361             num = len(clone)
  362             if num == 1:
  363                 return clone._result_cache[0]
  364             if not num:
  365                 raise self.model.DoesNotExist("%s matching query does not exist."
  366 ==>                     % self.model._meta.object_name)
  367             raise self.model.MultipleObjectsReturned("get() returned more than one %s -- it returned %s! Lookup parameters were %s"
  368                     % (self.model._meta.object_name, num, kwargs))
  369
  370         def create(self, **kwargs):
  371             """
  372             Creates a new object with the given kwargs, saving it to the database
Variables:
               clone = []
                 num = 0
                self =
[<ManagedObject: SAE>, <ManagedObject: as1-len19>, <ManagedObject: as1-rub26>, <ManagedObject: as1-rub36>, <ManagedObject: as2-rub26>, <ManagedObject: pe1-m9>, <ManagedObject: pe2-m9>, <ManagedObject: r1-avi13>, <ManagedObject: r1-bor7>, <ManagedObject: r1-nov7>, <ManagedObject: r1-yam3>, <ManagedObject: sw1-avi13>, <ManagedObject: sw1-bor7>, <ManagedObject: sw1-m9>, <ManagedObject: sw1-nov7>, <ManagedObject: sw1-pes10>, <ManagedObject: sw1-rub26>, <ManagedObject: sw1-rub36>, <ManagedObject: sw2-m9>, <ManagedObject: sw2-nov7>, '...(remaining elements truncated)...']
                args = ()
              kwargs = {'pk': 38}
------------------------------------------------------------------------
File: lib/python2.7/site-packages/django/db/models/manager.py (Line: 131)
Function: get
  124         def distinct(self, *args, **kwargs):
  125             return self.get_query_set().distinct(*args, **kwargs)
  126
  127         def extra(self, *args, **kwargs):
  128             return self.get_query_set().extra(*args, **kwargs)
  129
  130         def get(self, *args, **kwargs):
  131 ==>         return self.get_query_set().get(*args, **kwargs)
  132
  133         def get_or_create(self, **kwargs):
  134             return self.get_query_set().get_or_create(**kwargs)
  135
  136         def create(self, **kwargs):
  137             return self.get_query_set().create(**kwargs)
Variables:
                self = <django.db.models.manager.Manager object at 0x80b753ed0>
                args = ()
              kwargs = {'pk': 38}
------------------------------------------------------------------------
File: lib/nosql.py (Line: 268)
Function: __get__
  261                 # Document class being used rather than a document object
  262                 return self
  263
  264             # Get value from document instance if available
  265             value = instance._data.get(self.name)
  266             # Dereference
  267             if isinstance(value, int):
  268 ==>             value = self.document_type.objects.get(pk=value)
  269                 if value is not None:
  270                     instance._data[self.name] = value
  271             return super(ForeignKeyField, self).__get__(instance, owner)
  272
  273         def __set__(self, instance, value):
  274             if not value:
Variables:
               owner = <class 'noc.inv.models.interface.Interface'>
            instance = repr() failed
                self = <noc.lib.nosql.ForeignKeyField object at 0x80b9549d0>
               value = 38
------------------------------------------------------------------------
File: inv/models/link.py (Line: 117)
Function: _update_pop_links
  110         def object_links_count(cls, object):
  111             ifaces = Interface.objects.filter(managed_object=object.id).values_list("id")
  112             return cls.objects.filter(interfaces__in=ifaces).count()
  113
  114         @classmethod
  115         def _update_pop_links(cls, sender, document, target=None, **kwargs):
  116             for i in document.interfaces:
  117 ==>             for o in Object.get_managed(i.managed_object):
  118                     pop = o.get_pop()
  119                     if pop:
  120                         refresh_schedule(
  121                             "main.jobs", "inv.update_pop_links",
  122                             key=pop.id, delta=5)
  123
Variables:
              sender = <class 'noc.inv.models.link.Link'>
                   i = repr() failed
              kwargs = {}
              target = None
            document = repr() failed
                 cls = <class 'noc.inv.models.link.Link'>
------------------------------------------------------------------------
File: lib/python2.7/site-packages/blinker/base.py (Line: 267)
Function: send
  260                                 '%s given' % len(sender))
  261             else:
  262                 sender = sender[0]
  263             if not self.receivers:
  264                 return []
  265             else:
  266                 return [(receiver, receiver(sender, **kwargs))
  267 ==>                     for receiver in self.receivers_for(sender)]
  268
  269         def has_receivers_for(self, sender):
  270             """True if there is probably a receiver for *sender*.
  271
  272             Performs an optimistic check only.  Does not guarantee that all
  273             weakly referenced receivers are still alive.  See
Variables:
                self = <blinker.base.NamedSignal object at 0x808174610; 'pre_delete'>
              kwargs = repr() failed
              sender = <class 'noc.inv.models.link.Link'>
            receiver = <bound method ?._update_pop_links of <class 'noc.inv.models.link.Link'>>
------------------------------------------------------------------------
File: lib/python2.7/site-packages/mongoengine/document.py (Line: 454)
Function: delete
  447             :param write_concern: Extra keyword arguments are passed down which
  448                 will be used as options for the resultant
  449                 ``getLastError`` command.  For example,
  450                 ``save(..., write_concern={w: 2, fsync: True}, ...)`` will
  451                 wait until at least two servers have recorded the write and
  452                 will force an fsync on the primary server.
  453             """
  454 ==>         signals.pre_delete.send(self.__class__, document=self)
  455
  456             try:
  457                 self._qs.filter(
  458                     **self._object_key).delete(write_concern=write_concern, _from_doc_delete=True)
  459             except pymongo.errors.OperationFailure, err:
  460                 message = u'Could not delete document (%s)' % err.message
Variables:
                self = repr() failed
       write_concern = {'write_concern': {}}
------------------------------------------------------------------------
File: lib/python2.7/site-packages/mongoengine/queryset/base.py (Line: 426)
Function: delete
  419
  420             call_document_delete = (queryset._skip or queryset._limit or
  421                                     has_delete_signal) and not _from_doc_delete
  422
  423             if call_document_delete:
  424                 cnt = 0
  425                 for doc in queryset:
  426 ==>                 doc.delete(write_concern=write_concern)
  427                     cnt += 1
  428                 return cnt
  429
  430             delete_rules = doc._meta.get('delete_rules') or {}
  431             # Check for DENY rules before actually deleting/nullifying any other
  432             # references
Variables:
                 cnt = 0
                 doc = repr() failed
                self = repr() failed
            queryset = .. queryset mid-iteration ..
   has_delete_signal = True
    _from_doc_delete = False
call_document_delete = True
       write_concern = {}
------------------------------------------------------------------------
File: sa/wipe/managedobject.py (Line: 85)
Function: wipe
   78         log.debug("Wiping discovery id")
   79         DiscoveryID._get_collection().remove({"object": o.id})
   80         # Wiping interfaces, subs and links
   81         # Wipe links
   82         log.debug("Wiping links")
   83         for i in Interface.objects.filter(managed_object=o.id):
   84             # @todo: Remove aggregated links correctly
   85 ==>         Link.objects.filter(interfaces=i.id).delete()
   86         #
   87         log.debug("Wiping subinterfaces")
   88         SubInterface.objects.filter(managed_object=o.id).delete()
   89         log.debug("Wiping interfaces")
   90         Interface.objects.filter(managed_object=o.id).delete()
   91         log.debug("Wiping forwarding instances")
Variables:
                   i = <Interface: wiping-2: Gi 0/3>
                  ac = <class 'noc.fm.models.archivedalarm.ArchivedAlarm'>
                  db = Database(MongoClient('localhost', 27017), u'noc')
                 log = <noc.lib.log.PrefixLoggerAdapter object at 0x80ec98d90>
                   o = <ManagedObject: wiping-2>
------------------------------------------------------------------------
File: main/management/commands/wipe.py (Line: 103)
Function: wipe_managed_object
   96             """
   97             Wipe Managed Object
   98             :param o: Managed Object
   99             :type o: Managed Object
  100             :return: None
  101             """
  102             from noc.sa.wipe.managedobject import wipe
  103 ==>         wipe(o)
  104
  105         def get_user(self, u_id):
  106                 """
  107                 Get User by id or name
  108                 :param o_id: Object's id or name
  109                 :return: ManagedObject
Variables:
                self = <noc.main.management.commands.wipe.Command object at 0x80ec40210>
                wipe = <function wipe at 0x80ec58a28>
                   o = <ManagedObject: wiping-2>
------------------------------------------------------------------------
File: main/management/commands/wipe.py (Line: 47)
Function: handle
   40                     raise CommandError("Object '%s' is not found" % o_id)
   41                 objects += [o]
   42             # Wipe objects
   43             from noc.lib.debug import error_report
   44             for o in objects:
   45                 with self.log("Wiping '%s':" % unicode(o), True):
   46                     try:
   47 ==>                     wiper(o)
   48                     except KeyboardInterrupt:
   49                         raise CommandError("Interrupted. Wiping is not complete")
   50                     except:
   51                         error_report()
   52
   53         @contextmanager
Variables:
                self = <noc.main.management.commands.wipe.Command object at 0x80ec40210>
                args = ('managed-object', 'wiping-2')
                   m = 'managed_object'
                   o = <ManagedObject: wiping-2>
             options =
{'pythonpath': None, 'settings': None, 'traceback': None, 'verbosity': '1'}
             objects = [<ManagedObject: wiping-2>]
              getter =
<bound method Command.get_managed_object of <noc.main.management.commands.wipe.Command object at 0x80ec40210>>
        error_report = <function error_report at 0x8070ab320>
               wiper =
<bound method Command.wipe_managed_object of <noc.main.management.commands.wipe.Command object at 0x80ec40210>>
                o_id = 'wiping-2'
------------------------------------------------------------------------
END OF TRACEBACK
ERROR:noc.lib.debug:Writing CP report to local/cp/crashinfo/new/ca6e7207-7bc6-5f5a-b35d-6565755ffad7.json
done