python - Amazon SQS duplicated messages in queue -

-

this question has answer here:

i not experienced used in amazon sqs service. have read messages queue not own , process them making small database of information.

up until had code read messages in queue , process them. script running periodically.

however, observed amount of messages in queue has become large. when took 10000 sample messages observed around 6000 duplicates.

i puzzled sudden change in behavior (up until did not observe duplicate messages). queue never seems run out.

this code use read messages queue.

conn = boto.sqs.connect_to_region(     'myregions',     aws_access_key_id='myacceskey',     aws_secret_access_key='secretacceskey') q = boto.sqs.queue.queue(connection=conn, url='outputqueue')  rs = q.get_messages(10) all_messages = [] while len(rs) > 0:     all_messages.extend(rs)     print (len(all_messages))     rs = q.get_messages(10)

can explain why getting duplicated messages suddenly? not have permissions see how large queue is, how can messages in it? doing right?

after processing message queue need send notification message has been processed , should deleted. failure mean message sits in queue , re-fetched until such time reaches fetch limit , sent dead letter queue or expires.

sqs not guarantee uniqueness , can duplicates, can set visibility timeout prevent message being read period of time after has been retrieved e.g. minute or give time process message , delete queue. should avoid duplicates.

as deleting message iterate on messages, process them , run either...

conn.delete_message(q, message)

or

q.delete_message(message)


Comments

Post a Comment

Popular posts from this blog

Delphi XE2 Indy10 udp client-server interchange using SendBuffer-ReceiveBuffer -

-
i use delphi xe2 , indy10 udp protocol. can't receive server echo on client side if use receivebuffer method. got "socket error # 10040" although send small echo message server client. console application illustrate problem below. in advance. program project1; {$apptype console} {$r *.res} uses system.sysutils, idglobal, idbasecomponent, idcomponent, idsockethandle, idudpclient, idudpserver, idudpbase, idstack; type tudp_serv = class(tidudpserver) procedure udpsvudpread(athread: tidudplistenerthread; adata: tidbytes; abinding: tidsockethandle); end; var udpserver: tudp_serv; udpcl: tidudpclient; bsnd, brcv: tbytes; s: string; k: integer; //============================================================================== procedure tudp_serv.udpsvudpread(athread: tidudplistenerthread; adata: tidbytes; abinding: tidsockethandle); begin writeln(' server read: ' + tohex(adata, length(adata))); abinding sendto(peerip, peerport, adat...
Read more

Qt ActiveX WMI QAxBase::dynamicCallHelper: ItemIndex(int): No such property in -

-
i new windows programming. here goes code objiwbemlocator = new qaxobject("wbemscripting.swbemlocator"); objwmiservice = objiwbemlocator->querysubobject("connectserver(qstring&,qstring&)",qstring("."),qstring("root\\cimv2")); qaxobject* returnlist = objwmiservice->querysubobject("execquery(qstring&)", qstring("select * %1").arg(domain)); qaxobject* result = returnlist->querysubobject("itemindex(int)", 0); i getting error on runtime qt activex wmi qaxbase::dynamiccallhelper: itemindex(int): no such property in but itemindex method exists msdn says the itemindex method not work collections not contain swbemobjects, such swbemmethodset, swbemnamedvalueset, swbemprivilegeset, swbempropertyset, , swbemqualifierset. what selecting ? have tried iterating on collection ? plus need minimum windows vista itemindex method. hope helps
Read more

Enable autocomplete or intellisense in Atom editor for PHP -

-
i'm first time user of atom editor php code. atom can autocomplete built-in functions of php. possible make editor autocomplete custom function written in separate file? function.php <?php function printsomething() { echo "hello world"; } index.php <?php require_once "function.php"; printsomething(); // autocompleted atom editor while typing thanks in advance. there text file in %userprofile%.atom called snippets.cson. in there can add snippet. more this: http://flight-manual.atom.io/using-atom/sections/snippets/ but need think can use later on other files well. otherwise need copy , paste.
Read more