記一次離奇的TNS-12545TNS-12560TNS-00515

Starting CRON daemondone
Oracle 10g auto start/stop
Startup "USMTHLY" listener.

LSNRCTL for Linux: Version 10.2.0.3.0 - Production on 08-JUL-2014 17:05:49

Copyright (c) 1991, 2006, Oracle. All rights reserved.

Starting /users/oracle/OraHome10g/bin/tnslsnr: please wait...

TNSLSNR for Linux: Version 10.2.0.3.0 - Production
System parameter file is /users/oracle/OraHome10g/network/admin/listener.ora
Log messages written to /users/oracle/OraHome10g/network/log/listener_usmthly.log
Error listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=10.101.1.106)(PORT=1520)))
TNS-12545: Connect failed because target host or object does not exist
 TNS-12560: TNS:protocol adapter error
 TNS-00515: Connect failed because target host or object does not exist
 Linux Error: 99: Cannot assign requested address

#上面的錯誤信息了來自系統啟動日志，同樣出現在該實例的監聽器日志里。 

#故障環境
oracle@sysreportDB:/users/oracle> cat /etc/issue

Welcome to SUSE Linux Enterprise Server 10 SP4 (x86_64) - Kernel \r (\l).

oracle@sysreportDB:/users/oracle> sqlplus -v

SQL*Plus: Release 10.2.0.3.0 - Production

#服務器啟動后，檢查數據庫實例，2個實例都已成功啟動
oracle@sysreportDB:~> ps -ef | grep pmon | grep -v grep
oracle 4916 1 0 Jul08 ? 00:00:00 ora_pmon_USMTHLY
oracle 5605 1 0 Jul08 ? 00:00:00 ora_pmon_USBO2

oracle@sysreportDB:~> ps -ef | grep lsnr| grep -v grep #僅僅只有usbo2的監聽被啟動
oracle 5597 1 0 Jul09 ? 00:00:00 /users/oracle/OraHome10g/bin/tnslsnr listener_USBO2 -inherit

oracle@sysreportDB:~> oerr tns 12545
12545, 00000, "Connect failed because target host or object does not exist"
// *Cause: The address specified is not valid, or the program being 
// connected to does not exist.
// *Action: Ensure the ADDRESS parameters have been entered correctly; the
// most likely incorrect parameter is the node name. Ensure that the 
// executable for the server exists (perhaps "oracle" is missing.)
// If the protocol is TCP/IP, edit the TNSNAMES.ORA file to change the
// host name to a numeric IP address and try again.
oracle@sysreportDB:~> oerr tns 12560
12560, 00000, "TNS:protocol adapter error"
// *Cause: A generic protocol adapter error occurred.
// *Action: Check addresses used for proper protocol specification. Before
// reporting this error, look at the error stack and check for lower level
// transport errors. For further details, turn on tracing and reexecute the
// operation. Turn off tracing when the operation is complete.
oracle@sysreportDB:~> oerr tns 00515
00515, 00000, "Connect failed because target host or object does not exist"
// *Cause: The address specified is not valid, or the program being 
// connected to does not exist.
// *Action: Ensure the ADDRESS parameters have been entered correctly; the
// most likely incorrect parameter is the node name. Ensure that the 
// executable for the server exists (perhaps "oracle" is missing.)

#上面的三個錯誤號描述信息表明該故障與主機名或IP地址有關，需要檢查是否為有效的IP或主機名
#下面查看監聽器及TNSNAMES的配置
oracle@sysreportDB:/users/oracle/OraHome10g/network/admin> more listener.ora
SID_LIST_LISTENER_USMTHLY =
 (SID_LIST =
 (SID_DESC =
 (sid_name=USMTHLY)
 (oracle_home = /users/oracle/OraHome10g)
 )
 )

LISTENER_USMTHLY =
 (DESCRIPTION_LIST =
 (DESCRIPTION =
 (ADDRESS_LIST =
 (ADDRESS = (PROTOCOL = TCP)(HOST = 10.101.1.106)(PORT = 1520))
 )
 )
 )

SID_LIST_LISTENER_USBO2 =
 (SID_LIST =
 (SID_DESC =
 (sid_name=USBO2)
 (oracle_home = /users/oracle/OraHome10g)
 )
 )

LISTENER_USBO2 =
 (DESCRIPTION_LIST =
 (DESCRIPTION =
 (ADDRESS_LIST =
 (ADDRESS = (PROTOCOL = TCP)(HOST = 10.101.1.106)(PORT = 1504))
 )
 )
 )

oracle@sysreportDB:/users/oracle/OraHome10g/network/admin> more tnsnames.ora
USMTHLY=
 (DESCRIPTION=
 (ADDRESS=
 (PROTOCOL=TCP)
 (HOST=10.101.1.106)
 (PORT=1520)
 )
 (CONNECT_DATA=
 (SERVICE_NAME=USMTHLY)
 )
 )

USBO2=
 (DESCRIPTION=
 (ADDRESS=
 (PROTOCOL=TCP)
 (HOST=10.101.1.106)
 (PORT=1504)
 )
 (CONNECT_DATA=
 (SERVICE_NAME=USBO2)
 )
 )
#從上面的配置來看，配置無異常。
#下面查看本機host及IP信息，從下面的描述來看，主機host配置無異常
oracle@sysreportDB:~> more /etc/hosts
127.0.0.1 localhost

# special IPv6 addresses
::1 localhost ipv6-localhost ipv6-loopback

fe00::0 ipv6-localnet
ff00::0 ipv6-mcastprefix
ff02::1 ipv6-allnodes
ff02::2 ipv6-allrouters
ff02::3 ipv6-allhosts
10.101.1.106 sysreportDB.2goasp.com sysreportDB 

oracle@sysreportDB:~> hostname
sysreportDB
# Author : Leshami
# Blog : http://blog.csdn.net/leshami

#下面是其IP信息
oracle@sysreportDB:/var/log> ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:70:82:63
 inet addr:10.101.1.106 Bcast:10.201.1.255 Mask:255.255.255.0
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
 RX packets:5763651 errors:0 dropped:0 overruns:0 frame:0
 TX packets:802254 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:1000
 RX bytes:8102327746 (7726.9 Mb) TX bytes:88597013 (84.4 Mb)

lo Link encap:Local Loopback #有環路能正常TX,RX
 inet addr:127.0.0.1 Mask:255.0.0.0
 UP LOOPBACK RUNNING MTU:16436 Metric:1
 RX packets:92972 errors:0 dropped:0 overruns:0 frame:0
 TX packets:92972 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:0
 RX bytes:4993723 (4.7 Mb) TX bytes:4993723 (4.7 Mb)

#嘗試手動啟動監聽，手動啟動監聽無異常 
oracle@sysreportDB:/var/log> lsnrctl start listener_USMTHLY

LSNRCTL for Linux: Version 10.2.0.3.0 - Production on 08-JUL-2014 17:10:49

Copyright (c) 1991, 2006, Oracle. All rights reserved.

Starting /users/oracle/OraHome10g/bin/tnslsnr: please wait...

TNSLSNR for Linux: Version 10.2.0.3.0 - Production
System parameter file is /users/oracle/OraHome10g/network/admin/listener.ora
Log messages written to /users/oracle/OraHome10g/network/log/listener_usmthly.log
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=sysreportDB.2goasp.com)(PORT=1520)))

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=sysreportDB.2goasp.com)(PORT=1520)))
STATUS of the LISTENER
------------------------
Alias listener_USMTHLY
Version TNSLSNR for Linux: Version 10.2.0.3.0 - Production
Start Date 08-JUL-2014 17:10:50
Uptime 0 days 0 hr. 0 min. 0 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /users/oracle/OraHome10g/network/admin/listener.ora
Listener Log File /users/oracle/OraHome10g/network/log/listener_usmthly.log
Listening Endpoints Summary...
 (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=sysreportDB.2goasp.com)(PORT=1520)))
Services Summary...
Service "USMTHLY" has 1 instance(s).
 Instance "USMTHLY", status UNKNOWN, has 1 handler(s) for this service...
The command completed successfully

#通過上面的測試可以看出，監聽器能夠正常啟動也就是說監聽器的配置應該是OK的。

#另外一點發現，系統啟動日志里邊有127.0.0.2的環路，不知道怎么會有127.0.0.2
Setting up network interfaces:
 lo
 lo IP address: 127.0.0.1/8
 IP address: 127.0.0.2/8

oracle@sysreportDB:/var/log> ping 127.0.0.2 #環路127.0.0.2 OK
PING 127.0.0.2 (127.0.0.2) 56(84) bytes of data.
64 bytes from 127.0.0.2: icmp_seq=1 ttl=64 time=0.034 ms

#對于上面的問題實在是百思不得其解，就差使用sqlnet trace。
#下面嘗試將監聽器配置文件內IP地址使用主機名來代替，竟然成功鳥。
#如下2種方式都可以將監聽隨系統啟動而啟動，一個是主機名，一個是full主機名

#LISTENER_USMTHLY =
# (DESCRIPTION_LIST =
# (DESCRIPTION =
# (ADDRESS_LIST =
# (ADDRESS = (PROTOCOL = TCP)(HOST = sysreportDB)(PORT = 1520))
# )
# )
# )

LISTENER_USMTHLY =
 (DESCRIPTION_LIST =
 (DESCRIPTION =
 (ADDRESS_LIST =
 (ADDRESS = (PROTOCOL = TCP)(HOST = sysreportDB.2goasp.com)(PORT = 1520))
 )
 )
 )

#高興之余，查看系統日志，汗，還是有錯誤，這次的錯誤是TNS-12543，目標主機不可達。 
#懷疑是否hosts文件中是否存在異常字符，于是嘗試重建Hosts文件，重建后錯誤消息依舊如下。
Starting CRON daemondone
Oracle 10g auto start/stop
Starting Oracle10g: Startup "USMTHLY" listener.

LSNRCTL for Linux: Version 10.2.0.3.0 - Production on 09-JUL-2014 17:15:47

Copyright (c) 1991, 2006, Oracle. All rights reserved.

Starting /users/oracle/OraHome10g/bin/tnslsnr: please wait...

TNSLSNR for Linux: Version 10.2.0.3.0 - Production
System parameter file is /users/oracle/OraHome10g/network/admin/listener.ora
Log messages written to /users/oracle/OraHome10g/network/log/listener_usmthly.log
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=sysreportDB.2goasp.com)(PORT=1520)))

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=sysreportDB)(PORT=1520)))
TNS-12543: TNS:destination host unreachable
 TNS-12560: TNS:protocol adapter error
 TNS-00513: Destination host unreachable
 Linux Error: 101: Network is unreachable

#最終，此題暫時無解，沒有找到原因，不影響自啟動而已。